applied optimization with matlab programming

MAn.AB is a regislered lrademark of宵leMalhWorks， Inc.

This book is p巾 ledon ncid伽 paper.@

Copyrighl@2∞2 by John Wiley & Sons， New York. AII righls陀賠刊ed.

Published simultaneously in Cnnadn.

No pnrt of Ihis publiclllion may be reprodu回 d，stored in a問triev叫systemor 1悶nsmittedin nny form or byanym回 ns，eleclronic， mechanicaJ， pholocopying，問cording・scanningor otherwise， except as permittcd under Sections 107 or 108 ofthe 1976 United， Stalcs Copyrighl Acr， wirhoul either the prior writtcnp苦rmissionof Ihe Publisher， or outhori7.3lion through paymenl of the appropriule per-copy fcc 10 Ihe Copyrighl Clearance Cenler， 222 Ro揖 W∞dDrive， Dnnver百， MA01923， (978) 750・8400，fnx (978) 750・4744.Requ田 IS10 Ihe Publisher for permission should be nddressed 10 the Pemlissions Departmenl，

John Wilcy & Sons， Inc.， 605百 irdAvenue， New York， NY 10158-∞12， (212) 850-ω11，加 (212)850-6008， E-Mail: [email protected].

η首spublicalion is designed 10 provide accu問lennd authorilalive infomtalion in冊 E副甘 10山esubjecl mattercove問d.It is 80ld with the underslunding Ihal the publisher is nol engaged in rend自吋ngprofessional services. If professionnl ndvice or othcr expert assislance is民quired，出eservices of a compelent professional pe目。nashould be soughl.

Ubrary ofCongress Catalog;lIg;n P"bl;catiotl Data:

Venkolurom且n，P. 由

Applied oplimiZlllion with MATLAB~ Programming I P. Venkalar百mlln.p. com.

“A Wiley-Intersciencc publicnlion." ISBN 0-471剖 958・.5(c1oth : alk. pnper) 1. MnlhemalicaJ optimization--Dolll proc巴出ing・2.MATLAB. J. Title.

QA402.5.V42 2001 519.3-dc21

Prinled in Ihe Uniled Slales of Americn.

10987654321

2001026938

~拍泊当.，“、“由4幽以罰r;r:T，τ-一一-一一 ←

T 0 coexistence of peace and harmony between the different melodies of people everywhere

一一一一一6巾一←一十十一

CONTENTS

PREFACE

1 Introductlon

1.1 Optimization Fundamentals I 2

1.1.1 Elements of Problem Formulation I 4

1.1.2 Mathematical Modeling I 10

1.1.3 Nature of Solution I 16

1.1.4 Characteristics of the Search Procedure I 20

1.2 Introduction to MA11..As I 25

1.2.1 Why MATLAB? I 25

1.2.2 MATLAB Installation Issues I 26

1.2.3 Using MATLAB the First Time I 27

1.2.4 Using the Editor / 33

1.2.5 Creating a Code Snippet / 37

1.2.6 Creating a Program I 40

Problems I 44

xiii

1

2 Graphlcal Optlmizatlon 45

2.1 針。blemDefinition I 45

2.1.1 Example 2.1 I 46

2.1.2 Format for lhe Graphical Display I 47

2.2 Graphical Solution I 48

2.2.1 MATLAB High-Level Graphics Functions I 48

2.2.2 Example 2.I-Graphical Solution I 50

2.2.3 Displaying the Graphics I 53

2.2.4 Customizing the Figure I 54

2.3 Additional Examples I 56 vll

vlll CONTENTS

2.3.1 Example 2.2 I 56

2.3.2 Example 2.3 I 64

2.3.3 Example 2.4 I 73

2.4 Additional MA11..AB Graphics I 79

2.4.1 Handle Graphics I 80

2.4.2 Graphical User Interface I 81

2.4.3 GUI Code I 84

References I 91

Problems I 92

3 Llnear Programmlng

3.1 Problem Definition I 94

3.1.1 Standard Format / 94

3.1.2 Modeling Issues / 98

3.2 Graphical Solution / 107

3.2.1 Example 3.1 / 110

3.2.2 Characteristics of the Solution / II1

3.2.3 Different Solution Types I 114

3.3 Numerical Solution-the Simplex Method I 115

3.3.1 Features of the Simplex Method / 115

3.3.2 Application of Simplex Method / 117

3.3.3 Solution Using MA11..AB / 120

3.3.4 Solution Using MATLAB'S Optimization Toolbox I 123

3.4 Additional Examples / 124

3.4.1 Example 3.2-Transportation Problem / 124

3.4.2 Example 3.3-Equality Constraints and Unres凶ctedVariables / 130

3.4.3 Example 3.4-A Four-Variable Problem / 134

3.5 Additional Topics in Linear Programming / 138

3.5.1 Primal and Dual Problem / 138

3.5.2 Sensitivity Analysis / 148

Refer芭nces/ 151

Problems / 152

4 Nonlinear Programming


93

154

CONTENTS Ix

4.1.1 Problem Formulation-Example 4.1 / 155

4.1.2 Discussion of Cons釘aints/ 157

4.2 Mathematical Concepts / 159

4.2.1 Symbolic Computation Using MATLAB / 159

4.2.2 Ba~ic Mathematical Concepts I 162

4.2.3 Taylor's Theorem/Series / 169

4.3 Graphica1 Solutions / 171

4.3.1 Unconstrained Problem / 171

4.3.2 Equality Constrained Problem / 172

4.3.3 Inequality Constrained Problem / 173

4.3.4 Equality and Inequality Constraints / 174

4.4 Analytica1 Conditions / 175

4.4.1 Unconstrained Problem / 176

4.4.2 Equality Cons回 inedProblem I 179

4.4.3 Inequality Constrained Optimization I 186

4.4.4 General Optimization Problem / 191

4.5 Ex創nples/ 194

4.5.1 Example4.2/ 194

4.5.2 Example 4.3 / 196

References / 200

Problems / 201

5 Numerical Technlques-The One-Dlmenslonal Problem


5.1.1 Constrained One-Dimensional Problem / 204

5.2 Solution to the Problem / 205

5.2.1 Graphical Solution / 205

5.2.2 Newton-Raphson Technique / 206

5.2.3 Bisection Technique I 209

5.2.4 Polynomial Approximation / 211

5.2.5 Golden Section Method I 214

5.3 Importance of the One-Dimensional Problem / 217

5.4 Additional Examples I 219

203

5.4.1 臥ample5.2-Il1us回tionof Geneml Golden Section Method / 219

5.4.2 Example 5.3ー Two-PointBoundary Value Problem I 220

5.4.3 Example 5.4-Root Finding with Golden Section I 223

X CONTENTS

References I 225

Problems I 225

6 Numerlcal Technlques for Unconstralned Optlmlzatlon


6.1.1 Example 6.1 I 228

6.1.2 Necessary and Sufficient Conditions I 228

6.1.3 Elements of a Numerica1 Technique I 229

6.2 Numerical Techniques-Nongradient Methods I 230

6.2.1 Random Wa1k I 230

6.2.2 Pattem Search / 234

6.2.3 PoweJ1's Method I 238

6.3 Numerical T，配hniques-Gradient-BasedMethods I 241

'6.3.1 Steepest Descent Method I 241

6.3.2 Conjugate Gradient (F1etcher-Reeves) Method I 244

6.3.3 Davidon-FJetcher-PowelI Method I 246

6.3.4 Broydon-Fletcher-GoldfartトShannoMethod I 249

6.4 Numerical Techniques-Second Order I 251


6.5.1 Example 6.2-Rosenbrock Problem / 253

227

6.5.2 Example 6.3-Three-Dimensional Aow n飼raRo凶 ngDiskI 255

6.5.3 ExampJe 6.4-Fitting Bezier Parametric Curvcs I 258

References I 262

Problems I 263

7 Numerlcal Techniques for Constralned Optlmlzatlon 265


7.1.1 Problem Formulation-Example 7.1 I 266

7.1.2 Necessary Conditions / 267

7.1.3 Elements of a Numerical Technique I 269

7.2 Indirect Methods for Constrained Optimization I 270

7.2.1 Exterior Penalty Function (EPF) Me出odI 271

7.2.2 Augmented Lagrange Multiplier (ALM) Method I 276

7.3 Direct Methods for Cons住ainedOptimization I 281

7.3.1 Sequentia1 Linear Programming (SLP) I 284

7.3.2 Sequential Quadratic Programming (SQP) I 289

CONTENTS xl

7.3.3 Genera1ized Reduced Gradient (GRG) Method I 297

7.3.4 S句uentialGradient Restoration AJgorithm (SGRA) I 302

7.4 Additiona1 ExampJes I 307

7.4.1 ExampJe 7.2-FJagpoJe Problem I 307

7.4.2 ExampJe 7.3-Beam Design I 310

7.4.3 ExampJe 7.4ーOptima1Control I 313

References I 316

Problems I 316

8 Dlscrete Optlmlzation

8.1 Concepts in Discrete Programming / 320

8.1.1 Problem Relaxation I 321

8.1.2 Discrete Optimal Solution I 322

8.2 Discrete Optimization Techniques I 324

8.2.1 Exhaustive Enumeration I 326

8.2.2 Branch and Bound I 329

8.2.3 Dynamic Programming / 336


8.3.1 Exnmple 8.4--1 Beum Design I 341

8.3.2 Zero-One Integer Programming I 343

References I 348

Problems I 348

9 Global Optlmlzation


9.1.1 Global Minimum / 351

9.1.2 Nature of the Solution I 354

9.1.3 Elements of a Numerica1 Technique I 356

9.2 Numerical T，白hniqu巴sand Additional Examples I 357

9.2.1 Simulated Annealing (SA) I 358

9.2.2 Genetic Algorithm (GA) I 366

References I 377

Problems I 378

10 Optlmization Toolbox from MATlAB

10.1 The Optimization Toolbox I 380

318

350

379

xll CONTENTS

10.1.1 Programs I 380

10.1.2 Using Programs I 382

10.1.3 Setting Optimization Parameters I 384

10.2 Examples I 385

10.2.1 Linear Programming I 385

10.2.2 Quadratic Programming I 386

10.2.3 Unconstrained Optimization I 388

10.2.4 Constrained Optimization I 389

Reference I 391

Index 393

PREFACE

百lesu~ject of optimization is receiving serious attention仕omengineers， scientists， managers， and most everybody else. This is driven by compe“tion， quality assurance， cost of production， and finally， the success of the business ente中市e.lgnoring出e

practice of optimization is not an option during current times.

Optimization is practiced through softwar芭 programsand requires significant computer resources. The techniques of optimization have not changed significantly in

陀centyears， but tlle areas of applications have mushroomed at a significant rate.

Successfully embedding the use of optimization in professional practice requires at least three P陀巴quisites.τbeyinclude mathematical modeling of the design problem， knowledge of computer programming， and knowledge of optimization techniques. Many special-purpose optimization software packages白紙 relax由巳 requiredknowledge of programming are available today. To use them efficiently， the remaining two areas still have to be addressed.

百lereare several excellent books on the subject of optimization， a few of them released recently. Most of them cov巴rthe subject in depth. which is necessary because

th巴mathematicalmodels are nonlinear and require special techniqu巴sthat are usually

notpaはofany core curriculum. All of山ebooks assume白紙出ereader is familiar wilh

a programming language，町aditionallyFORTRAN， and recently， C. Another assumplion h芭quentlymade is白紙 theoptimization包chniqueswi1l be implemented in a

mainframe computing environmenl. Such a combination is ex官emelydifficu1t for the

self-Iearner even with enormous motivation. An explosion of inexpensive desktop computing resources aids engineering design

practi印 today.Paralleling this development is出eavailability of extensive陀 sou陀凶

on the Internet， both for learning and deployment. This is joined by the availability of sof抑制官systemsthat provide叩 opportunityfor handling mathematics， graphics. and programming in a consistent manner. These software systems are significant1y easy to

master compar吋 tothe higher-level programming languages of the p陀viousye町 S.

百lisbook primarily seeks to harness this釘iangulationof s巴rvicesto provide a practical approach to the study of design optimization.百lebook uses MATI.AB@ to

i11ustrate and implement the various techniques of optimization. MATLAB is a product

from MathWorks. Inc. This book attempts 10 accomplish two important obj田 tives.The fir

xlv PREFACE

of symbolic. numerical. and graphical fl回 turesofMATLAB・ltintegrates山ispowerful

combination during恥凶nslationof many algorithms into app1ied numerical techniques for design optimization. There is a constant enhancement of the

programming skills throughout th巴book.The second 0吋ectiveis th巴primarytask of

侃 book-tocommunicate and demonstrate various lIumerical techniques that are

currenlかusedin tlte area oloptimal des刷.AH of thc numerical techniquesMEsupported by MMLAB code availableぉ

computer files. These files are available on a companion web site

www.wilev.comlvenkat.ltis necessary to visit the web site to download thc files to

fo l l o w a l l of t h e E X tl I n p i es -Th e b oo k w i l l b e e 蹴 n削tia叫llyin脚c∞omple巴t旬.ewiωtho削削0肌凶u凶山』此tthcsc files.

At the web site you will also find uscful links 10 other resources， a web course on

MATLAB programming，and updated information about Applied Optintimuon with

MATl.AB@ Programming. The decision to locate the files on a web site pmvidEs an avenue to shadow the

dynamic changes in MATLAs sof1ware itsclf， which is bcing transformed in a major

way almost every ycar.lt provides a mechanism 10 correct bugs i日出ecode in a timely

manner. It estab¥ishes an opportunity to improve自己codethrough reader suggcstions

and provides a way for出eauthor 10 keep thc rcader engaged with new dcvclopments.

It doωcut down the sizc of the book enormously by locating all of the pag回 ofcode

outside the book. An unusual feature of the book is由cinclusion of discrete optimization and global

optimization. Tradilional continuous design optimization techniques do take up a

significant portion of the book. Continuous problems generally allow the algorithms

to be mathematically developed in a convincing manncr， and由民eideas could be向core of original a1gorithms dcvc10pcd by the readers to address their own special

necds. Every chaptcr includes additional nontrivial cxamp1es由atoftcn prescnt a

novel use or extcnsion of thc optimization techniques in the chapter. The mathcmatical

algorithms and cxamples ar芭 accompaniedby MA'礼 ABcode available at the

companion web sue-Thenumerical techniques usually include strong graphical

support to il1ustrate出copcration of thc techniques. Discrete andglobal optimization techniques are basedon verydifferentalgorithms，

and回 .chof thcm can easily justify full-fledged independent courscs de叩tcd

exclusively 10 thcir S1udy. In a comprchensivc book such

PREFACE XV

classroom setting withou1 computational cxperience is also feasib1e. In由iscase the

a1gorithms can be presented and the nurnerical results illustrated. The senior/gradua1e

s1udents in various discip¥ines. especially engincering， are the targc1 audience. Optimization techniqucs are an importanl 1001 to solve dcsign problems in all

professional areas of study. Many illustrations are仕omthe area of mechanical

engineering reflecting出巴experienceof白eau白or.

Independent learnel's， particuIarly professionals who need to understand the subject of optimization， should also find the book very useful. A reader who has access to

MATLAB software c剖 us巴 thebook to its best advantage. The book is largely

self-contained and develops all ncccssary mathematical concepts when ncedcd.

Abstract mathematical idcas of optimi国 tion創芭 introduccdgraphically and through

iIlustrative examples in thc book. In many instances， the operation of thc numcrical technique is祖国atcdto show how the design is changing with iterations. This has

been possible because MATLAB graphics are rich in fcatures and simple to inco叩oratc.

making Ulcm cffcctive for convcying出eidcas. It is for this reason出ereader is

cxposed to graphical programming early on， incIuding a chapter on graphical

optimization. To balance thc development of programming skills and th巴presentationof optimization concepts， programming issues are incrementally explored and

implemented. Investment of time in d巴velopingprogramming skills is出emost effective way to imbibe them. Thc author is nOl aware of shortcuts. Studcnts arc

expected to program independently. make mistakes， debug errors. and incorporate improvements as p紅 tof their learning experience delivered through出isbook.

The book proceeds at a brisk pace to provide experiencc in MATLA8 programming

and communicate ideas in optimization in a reasonable number of pages. This could

not be possible without taking advantage of Ihe accompanying web site 10 locate much

of the code for algorithms and examples. Merely running the code will n01

significantly e油田cethe reader's programming skills. It is important for the student

to understand the code also. To箇 sistin白isprocess， the code carries Iiberal comments often corresponding to the algorithm that is being implemcnted. The au出orexpects

出ereader to have read出ecomments. The accompanying code allows the instructor

to assign both programming assignments as well as so

xvl PREFACE

approaches， the instructor can choose to include only a subset of continuous

optimization algorithms to allow the inclusion of出esepopular techniques. If出e

student has access to the Optimization Toolbox from MATlAB，白enit can be integrated

into出ecourse for handling the final design project. The last chapter provides an

overview on the use of the toolbox. This book started out using MATI.AB Version 5. About halfway through the book

血etransition to MATlAB Version 5.3 was complete. At the completion of the

m叩 uscriptMATLAB Version 6 wωshipping for some time. As of this tirne， Version 6.1 is available. Most institutional MATlAB licenses are usually based on subscription so白紙theywill have出巴 latestversion of白esoftware. In a sense， and出isis true of

all books that depend on particular software systems， the book rnay appear out of date with respect to白esoftware. This is not really so. While the usage ofMATLAB functions

rnay change between versions， there should be suitable wamings and instruction about

出edifferent usage・Animportant insurance is built into the book， particularly to handle such changes if they arise. Sprinkled throughout the book is the habit of由e

using出eonline help. In most instances出iswas primarily done to expose the reader

to alternative use of the same command. Another significant change the manuscript

had to endure was白eloss of access to a UNIX version because of th巴university's

change to distributed computing on PC clusters. The author's experience wi白血eearly chapters sugg巴s18that this should not mat1er. Except for external files and

resources，也巴 useof MATlAB as illustrated in the book is indifferent to the various

operating sy自由 (Windows，Mac， UNIX， or Linux). The topics chosen for inclusion and the simplicity of presentation of出etopics are

directly related to the experience of teaching the course on optimization at the senior/graduate level in出eD巴partmentof Mechanical Engineering for over ten years.

Experience proved白atcomprehension irnproved with simple iIIustrations， however complicated the mathematical ideas. On the other hand， straighぜorwardmathernatical expressions elicited no interest or understanding. The emphasis on application was

important to keep the students' attention. In this connection the author would like to 出創1kall his studen18 for pushing hirn for simple explanations， for providing positive interaction， and for their contribution of original and useful ideas in the discuss

PREFACE xvii

炉羽山dllpdLY

』

The book was made possible through support from John Wiley叩 dSons，Inc.，叩dMathWorks， Inc. Sincere thanks are owed to Bob Argentieri， senior editor at Jobn Wiley for accepting the proposal，and who all along displayed a lot of patiencein getting the book moving forward.Same is due to Bob Hilbed-associate managing editor at John Wiley， for his imp陀ssivework at c1eaning up the manuscript. Brian Snapp， New Media editor at John Wiley， created出巴 companionweb site (www.wiley.comlvenkat) and w辿 alsobe rnaintaining it. Naomi Femandes企omMathWorks， Inc.， saw to it白at1 had the latest version of MATLAB as soon as it was available. My regard for Dr. Angelo Miele， Professor Emeritus， at Rice University is more than can be expressed in these Iines. It was he who introduced me to 1h巴subjeclof optimization and demonstrated the effectiveness of simple presenution-I will always regard him as a great teacher. Of course， my family deserves special rnention for putting up with all the “Not now，"“Later，"“How about tornorrow?" d町in2debugging白ecode. Sp配 ial出回ksare to A帥 ana阻 dVinayak， rny offspring， ti町their patience， understanding， and encouragement. The au白orapologizes for any shortcornings on the presentation and welcomes comments， criticisms， and suggestions for improvement at all times.

P. VENKATARAMhN Rochesteれ NewYork

1・・-・a回.. 圃・・・・・・・tJFJ一J-

むす

pdrhw主にAbrψr=353thtY4y

INTRODUCTION

Optimization has become a necessary part of design activity in all major disciplines. These disciplines are not restricted to engineering. The motivation to produce economically relevant products or services with embedded quality is the principal reason for由isinclusion. Improved production and design tools， wi出 asynergistic thrust through inexpensive computational resources， have aided the consideration of optimization methods in new developments， particularly engineering products. Even in the absencc of a tangible product， optimization ideas provide the ability to define and explore problems while focusing on solutions出atsubscribe to some measure of usefulness. Generally， the use of the word optimization impli巴sthc best result under th巴circumstances.This includes the particular set of constraints on the development rcsources， current knowledge， market conditions， and so on. Every one of us has probably used thc term at some time to describe出eprimary quality of our work or endeavor. It is probably山emosl used or abused lerm in advertising and presentations. Neverthcless，白cability to make the best choice is a perpelual desire among us all.

Optimization is frequently associated with design， be it a product， service， or strategy. Aerospace design was among出巴earliestdisciplines to embrace optimization in a significant way driven by a natura1 need to lower the tremendous cost associated with carrying unnecess町yweight in aerospace vehicles. Minimum mass structures arl巴山 norm.Optimization forms part of the psyche of every aerospace designer. Saving on fuel through tr司jectorydesign was another problem that suggested itself. Very soon 曲巴enti問 engineeringcommunity could recognize由cneed to define solutions based on merit. Recognizing the desire for optimization and actua1ly implementing we陀 twodifferent issues.

2 INTRODUCTION

Until recently. for much of the time. optimization was usually attempted only in

those situations where there were significant penalties for generic designs. The

application of optimi却 tiondemanded large computational resources. In the nascent

years of digital computation these were available only to large national laboratories

and research programs. These resources were necessary to handle th巴 nonlinear

problems出at創芭 associatedwith engineering optimi回 .tion.As a result of these

constraints most of the everyday products were designed without regard to

optimization. This includes everything you s回ぽoundyou or use in your daily life. It

is inconceivable出atthe new generation of replacement products. likc the car. the

house. the desk. or出epencil. redesigned and manufactured today arc not designed

optimally in one sense or anothcr. Today. you would defmit巴Iyexplore procedures to optimize your inves回lentsby

tailoring your portfolio. You would optimize your business travel time by

appropria旬Iychoosing your dcstinations. You can optimize your commuting tim巴by

choosing your time and route. You can optimize your necessary expenditure for Iiving by choosing your day and store for shopping. You can optimizc the useful time you

connectto由巳Intemetby detcrmining your time of connection. You can buy software

白紙 willoptimize your conncction to thc lntemct. You can buy books or read articles

白紙 tcllyou how to perform these various optimizations. The above activitics

primarily relatc to :;crvIces or strategy. It is now apparcnt白atevery activity， except a巴S出etic，provides the scope for optimization.百lisjustifies looking at白estudy of

ootimiz副onas a tool白紙canbe applied 10 a variety of disciplines. If so， the myria叫d

ofo叩矧刑p戸凶tim凶iz捌a悦叫t“io叩n吋叩p仰r叩ob愉le巴m附ns凶s合伽伽伽oω叩11削I

Thisiおs曲eemphaおsiぬsoft由hebook. The partnership between d巴signand optimization activity is often found in

engineering. This book recognizes that connection and many of thc problems used for

illustrations and practice are from engin田 ring，primariIy mechanical， civil， and

aerospace design. Ncvertheless the study of optimization. particularly applied

optimization.is not m exclusive pmperty ofmy sptcinc discipurle.It involves the

dECOVEryanddesignof叫 utionsthrough appropriate techniques鰯 ociatedwith the

formulation of the problem in a sp田 ificmanner. This can be done for example in

economics， chemistry， and business man

1.1 OPTIMIZATION FUNDAMENTALS

Optimization can be applied to all disciplincs. Qualitatively， this assertion implies

multiple decision choices; implicitly recognizing the n巴cessityof choosing among

a1tematives. This books deals with optirnization in a quantitative way. This means that

an outcome of applying optimization to曲eproblem， design. or service must yield

number富山atwill define the solution. or in other words， numbers or values白紙 will

characterize the particular design or service. Quantitative dcscription of出巴回lution

requires a quantitative description of the problem itself. This description is called a

;:.~ ".

"

1.1 OPTIMIZATION FUNDAMENTALS 3

mathematical model.τbedωi伊.iωcharactcrization.四 dits circumstances must be

expressed mathematically. Consider the design activity in the following cases:

• New consumcr research indicatωthat people like 10 drink about 0.5 litcr of soda

pop al a time during the summer months. The fabrication COSI of the redesigned

soda can is proportional to出巴 surfacearea. and can be estimatcd at $1.00 pcr

square meter of the materiaI used. A circular cross section is the most plausible

given current tooling available for manufacture. For aesthetic reasons，白e

height must be at least twice the diametcr. Studies indicate出atholding comfort

requires a diameter between 6 and 9 cm.

• A cantilever beam nceds to be dcsign巴dto carry a point load F at the cnd of a

beam of length L. The cross section of the beam wiII be in the shupe of th巴l巴tterI(路島町edωas却 I-b回 m).The beam should meet prescribed failure criteria.

百 ereis also a limit on iωdefI田tion.A beam of minimum mass is requir吋 to

be designed.

• MyPC Company has decidcd to invcst $12 million in acquiring several n巴W

Componenl Placem巴nl Machines to manufacture differcnt kinds of

mothcrboards for a ncw gcncration of personal compulers. Three models of

thcsc machines are under consideration. Total numbcr of operators available is

100 becausc of the local labor market. A fIoor spacc constraint needs 10 bc satisfied because of the diffcrent dimcnsions of thcse machines. Additional

information rclating to each of the machines is given in Table 1.1. The company

wishes to determine how many of each kind is appropriate 10 maximize the

number of boards manufactured per day.

The above list rcpr凶 entsthree problems山atwiII be used 10 definc formal elements of an optimization problem. Each problem陀quiresinformation from the specific arca

or discipline to which it refers. To rccognize or design these problems assumes出at

出巴 designcris conversant wi白山eparticular subj田 tmatter.百leproblems are kept

simple to focus on optimization issucs. Problems of such variely are important today.

Recent advertisements in general consumer magazines i1Iustrate山atAlcoa (an

aluminum m卸 ufaclurer)is quit巴happyto have rcducedLhc weight of the standard

soda pop can by over 30% in recenl years. A similar c1aim by the plastics industry with respect to the standard milk jug (gaIlon) is al50 evident in these magazines， although in出iscase the number is 409る.Roof coUapses in白cNortheast due 10 excessive snow

Table 1.1 Component Placement Machir悶

Machine Operablc Model BoardTypes BoardslHour Opemtors/Shift HoursIDay CostIMachine

A 10 55 18 400，000 B 20 50 2 18 600，000 C 18 50 2 21 700，000

4 INTRODUCTION

during the 1998-1999 winter will have structura1 designers and homebuilders

exploring白巴 secondproblem. Th巴vibrantstock market in these times has made

ba1ancing the investment portfolio more challenging. Thc third case may suggest a

mathematica1 model appropriate for such decision making.

1.1.1 Elements of Problem Formulatlon

In this section， we will in凶 duce出eforma1 elements of thc optimization problem. In this book， the tcrm product a1so問fersto a servic巴 ora strategy. It should be

understood出atop自凶zationpresupposes the knowlcdge of由edesign rules for the

sp配 ificproblem， primarily the ability to describe the design in mathematica1 terms. These terms泊cJudedesign variables， design parameters， and design functions.

Traditiona1 design practi田，出atis， design without regard to optimization， incJudes a11

of these elements剖伽ugh血eywere not formally recognized as such. This also

justifies出eprerequisi飽 thatyou must be capable of dcsigning the 0吋∞tif you are

planning to apply出etechniqucs of optimi四 tion.It is al80 a good idea 10 recognizc

that optimization is a procedurc for scarching 1he best design among candidat巴s，cach of which can produce an acceptable product. The oced for thc objec1 or product is not

questioned here. but this may be due to a decision based 00 optimization applied in another disciplinc.

Deslgn Varlables: Design variables are entities that identify a particular design. In 出esearch for白巴 optimaldesign，出eseentities will change over a prcscribed range.

The values of a complete set of these variables charactcrize a specific design. Th巴number and type of cntities bclonging to this set are vcry important in identifying and

setting up出equ姐 titativedesign problem. It is essential出鉱山ischoice captu関白e

essence of山 objectbeing desi伊 ed叩 dat出esamc time provide a quantitative

characterization of the design problem. In applicd mathematical terminology. design variables serve as the unknowns of the problem being solvcd. Borrowing an ana10gy from the area of system dynami回 andcontrol theory，出eyare equiva1ent to defining the state of the systcm， in由iscase， thc statc of design. Typically， design variables Can be associatcd with describing出巴 object'ssize such as its length and height. In other

cぉes，they may represen1 the number of items. The choice of design variables is the

responsibility of由edesigner guidcd by intuition， expe凶se，and knowledge.百le民 is

a fundamenta1 requirement to be mct by山isset of design variables， namely，山eymust

be Iinearly independent. This means出a1you cannot cstablish th巴valueof onc of出e

design variables from山evalues of出eremaining variables through basic arithmetic

(scaling or addition) operations. For example， in a design having a rcctangular cross

section， you cannot have由民evariables reprωenting白elcng出，width， and area. If the

first two ar巴 prescribed，出ethird is automatica1ly established. In complex designs，

these relationships may not be very apparent. Nevertheless， the choi田 ofthe set of

design variables must meet由巳 criterionof linear independence for applying the

techniques of optimization. From a practicnl pcrspcc1ivc， thc property of Iinear independence identifies a minimum set of variables that cun complctely describe the

r::.-: 砕 l

'" ち

ι


design. This is significant because the effort in obtnining the solution varies as an

integer power of the number of variables， and this power is 1ypica11y greater山an2.

M回 tingthe requirement ensures reduced difficulty in mathematically exploring thc

solution.

百leset of design variables is identified as the design vector.百1Isv田 torwill be

consider吋 acolumn vector in this book. In fac1， a11 vectors are column vectors切出E

text. The length of血isvector， which is n， is the number of design vari油lesio曲E

problem. The design variables can cxpress different dimensional qu叩 titiesin出巴

problem， but in the malhematical model， they are distinguished by山巴charac1erx. AlI of the techniques of optimization in this book nre bascd on出eabstract mathematical model.百lesubscript on x， for cxamplc， X3・rcpresentsthe third design variable， which

may be the height of an objec1 in白ech釘ucterizationof the product. This abstract

model is ne回 ssaryfor mathematical convenience. This book will refer to曲edesign

variables in one of出巴 followingways:

(1) [羽-refc町ingto the vcc10r 01' d巴signvariablcs

(2) X or x-referring to thc vcctor again， omilling 1hc square brackets for convcnicncc if appropriate

(3) [xj， x2'・...，XIl]T_ーindicatingthc vcclor lhrough its elements. Nole山e

superscript transposition symbol to identify it as a colunm vector.

(4) X;. i = 1， 2. . . . ， n-referring to all of the elements of the design v巴ctor.

Thc above notationa1 convenience is extcndcd to all vcctors in the book.

Deslgn Parameters: In出isbook，山eseidentify constan岱伽1will not change凶

di釘erentdesigns are compared. Many 1exts use出巴 termdesign parameters 10 問prescnt白巴dcsignvariables we defined earlier and do not formally recognize design

parameters as defincd hc問.The principnl reason is that parameters have no role 10 play

泊 deten凶ning由巳optima1design. They are significant in the discussion of modeling issues. Examples of parameters include matcria1 propc民y，applied loads. and choicc of shape. The paramctcrs in the abstract mathematical model are rep隠 sentcdin a

similar form as the design vec1or， excep1伽 twe use the charactcr p. Therefore. [P]， P， [P"P2'・...Pq1 rep隠 sent曲eparameters of the problem. Note白紙白eleng白 ofthe

par副ne飽rvector is q. Except in山cdiscussion of modeling，白epar沼neterswill not be explicitly 児島rredto， as they are primarily predet巴rmincdconstants in白巴 design.

Des句'nFunctlons: Design functions define meaningful information about the design. They are evaluated using the design variables and design parameters discussed earlier. They establish the mathematical model of the design problem. These functions can represent design objective(s) andlor constraillts. As iωname implies， design 0肖ectivedrives the search for the optimal design. The satisfaction of the

印刷trailltsestablishes the validity of the design. If not explicit1y stated，出edesigner

is responsiblc for id巴ntifyingthc objectivc and constraints. Minimize the mass of the

6 INTRODUCTION

structure will translate to叩 objectiveρU1ction.The 由民sin the material must be less

than the yieId strength wi1l translate to a constraintfunction. In many problems， it is possible for thc same function to switch roles to provide diffcrcnt design scenarios.

ObJectlve Function向井 Thetraditional design optimization problem is dcfin巴dusing a single objective function.百leformat of this statemcnt is usually to minimizc

or maximize some quantity白紙 iscalculated using somc dcsign function. This

function must depend， explicitly or implicitly， on the design variables. In the literaturc，出isproblcm is cxp問ssedexc1usively， without loss of generaIity， as a

minimum problem. A maximum problem can b巴rccastas a minimization problcm

using the ncgativc or出巴reciprocalof thc function used for the objectivc function. ln

the first example in加 ducedearlier，出co句ectiveis to minimize cost.百lerefore，恥

desi伊 functionrepres印刷gcost wilI be the objective function. In由esecond伺 se，白eobjective is to minimize mass. In the third case， the objective is to maximizc machine utilization.百lear国 ofsingle objective desi伊 isconsidered mature today.

No new solution techniques for classical problcms hav巴becnadvanced for somc time

now. Today. much of the work in applied optimi回 tionis dirccted at expanding

applications to practical problems. ln many cases. this has involved crcative use of the

solution techniques. In由巳 abstractmathematical model. the objective function is reprosented by the symbolf To indicate its dependence on the design variables. it is f陀 quentlycxpressedぉ f(x"x2.. . . • xn). A more concisc陀 prcsentationis j{X). Single objective problcms have only one function dcnoted by f It is a scaIar (not a

V民 tor).Notc山atalthough the objcctive function depends on P (parameter vector). it is not explicitly inc1uded in the format.

Multiobjective and multidisciplinary d巴signsarc important developments today. Multiobj∞tive design， or multiple objecti ve design. refersωusing sevcraI di仔erentdesign functions to命的巴曲es田町hfor op曲naldωign. GeneraIly，山eyare expec舵dωbe

conflicting 0同cctives.They could also be∞opcrating objectives.百lecurrent approach to出esolution of these problems involves standard optimization proced山首 applicdto a

single (田onstructedobjective optimization problcm based on the different multiple

objectives. A pop叫arapproach is to use a suitably weigh包dlin伺 rcombination of出巴multiplc objectives. A practicallimitation wi白血isapproach is也.echoi回 ofweights us吋

in the model. This approach has not been cmbraced widely. An altemative approach of recognizing a prcn抑制ective，and solving a singlc objectivc problem with additional

constraints based on血児問ningo同制ve釦nctionscan usually gene峨岨accep帥 le

solution. In multiobjective problen

llY244円仁

yfJ


Constraint Functlons: As design functions， these will be influenced by the design variables. The format of these functions requires them to be compared to some

numerically limiting value山atis established by design requirement， or the designer. This value remains constant during thc optimization of the problcm. A wcl1-describcd dcsign problem is expected to inc1ude scveral such functions. which can be represented as a vcctor. The comparison is usual1y set up using the threc standard

叫甜onaloperators: =， S， andミ.Consider our first example. LetfimJ(X) represent

出efunction白紙 calculatesthe volume of the new soda can we are designing.百le

constr凶nton the d巴signcan bc expresscd as

ルIIJ(X)= 500 cmJ

In tlIe second example.letβm2(X) b巴thefunction that calculates the deflection of the

beam under the applied load. The constraint can be statcd as

ft11l2(X) :S; I mm

Thccons回 intfunctions can be c1assified as eqllality constraints [likefunl(X) aboveJ or inequality cons町出nts[likefun2(X)].

Problems without constraints are tcrmcd unconstrained problems. If constraints are present， then meeting them is more p創司mount山anoptimizalion. Constraint

satisfaction is necessary beforc the dcsign established by白巴 currcntvalue of the

desi伊 variablesis considered valid and acceptable. If constraints are not s姐sfied，then there is no solution. A feasible design is one in which all of the constraints are

satisfied. An optimal solution is one出athas met the design objective. An optimal

dcsign must be fcasiblc. The design spacc cncloscd by thc constraints is called the

feasible domain. Dcsign space is described a few paragraphs bclow.

Equallty Constralnts: Equality constraints are mathematically neat and easy to handlc. Numerically，出eyJ1叫uiremore effort to satisfy. They are also more restrictive

on tlIe designωthey limit the region from which the solution can be obtained. The

symbol representing equality constraints in the abstract model is h. There may be more than one equality constraint in the design problcm. A vector representation

for叫ualityconstraints is introduced through the following representation. [8]， [hl， h2， . . . . h，]. and hk: k = 1，2， . . . .1 are ways of identifying出eequality constraints.百le

dependence on the design variables X is omitted for convcnience. Notc出atthe length

of the vector is 1. An important陀箇onfor distinguishing the equality and inequality

constraints is山atthey arc mnnipulated differently in the senrch for the optimal

solution. The number n of design variables in出cproblcm must be greatcr出叩 the

number of equality constraints 1 for optimization to take place. lf n is cqual to 1. then

恥 problemwill be solved without reference to the objective. ln mathematical terms

the number of equations matches白enumber of unknowns. If n is less than 1， then you have an overdetermined scl of relations which could result in an inconsistent problcm

definition. The sct of cquality constraints must be linearly indcpendent. Broadly， this

8 INTAODUCTION

implies白紙 youcannot obtain one of the cons町aintsfrom elementary arithmetic operations on the rcmaining constraints. This serves 10 ensure that the mathematical search for solution will not fail. These tcchniques are based on methods from Iincar algebra. In the standard format for optimization problems， the叫ua1itycons町aintsare written with a 0 on the right-hand side. This means白紙山eequality constraint in thc first example will be exp陀ssedas

hl(X):funl(X) -500 = 0

In practica1 problems， equality constraints are rar官Iyencountcred.

Inequallty Constraints: In叫ua1itycons釘aintsapp回 rmore naturally in problcm formulation. Inequality constraints also provide more flexibility in dcsign selection. The symbol represcnting inequality cons釘'aintsin the abstract model is 8. There may be more出叩 oncin珂ua1ityconstraint in the design problem. The vector repr巴sentationfor inequality constraints is similar to what we hav巴seenbefore. Thus， [G)， [81，82， . . .・ 811/]， andめ:j = 1， 2， . . . ， m arc ways of identifying Ihe inequality cons甘aints.m reprωents出enumber of inequa1ity constraints. AIl design functions explicitly or implicitly depend on the design (or independent) variable X. 8 is used to descriho bo出 lossthun or cqual 10恒)朗dgrealcr白山 01"equlll 10 (と)constralnts. Strictly greater出叩(>)and strictly Icss th聞(<)are not used much in optimization because出巴 solutionsare凶 ua11yexpected to lie at出econstraint boundary. In the standard format， all problems arc expressed wi自由eS relationship. Moreover， the right-hand side of出eSsign isO.百leinequality constraint from the second example ftm2(X) is set up as

81(X):ルn2(X)-1三O

In由巳 caseof inequality constraints a distinction is madc as to whether the dcsign variables lie on the constraint boundary or in出巴interiorof the region bounded by白E

constraint. If the set of d巴signvariables lie on the boundary of白econstraint， mathematically， this expresses出efact that constraint is satisfied with strict equality，白紙 is，g = O. The constraint acts like an equality constraint. In optimization terminology，血isparticular cons住aintis referred to as an active constraint. If the set of design variables do not lie on the boundary， that is， they li巴inside出eregion of thc constraints， they arc conside陀dinactive constraints. Mathematica1ly，出econstraint satisfies出erclation g < O. An inequality consU'aint can thercfore be either active or lllactJve.

Slde Constralnts: Side cons釘aintsare a necessary part of the solution techniques， es戸cia1lynumerical oncs. They express the range for the design variablcs. Each design variable must be bound by numcrical va1ues for its lower and upper Iimit. The designer makes this choice based on his anticipation of an acceptable design.

4自

F 1.1 OPTIMIZATION FUNDAMENTALS 9

Design Space:ηle design space，出espace白atwi¥l be searched for optimal design， is the Euclidean or Cartesian n-dimensional space generated by the n independent design variables X. This is a genera1ization of the three-dimensional physical space with which we are familiar. For ten d巴signvariables， it is a ten-dimensional space. This is not easy to imagine. Il is a1so not casy to express this information 白rough a figure or graph because of the Iimitation of the three-dimensional world. However， if也ed巴:signvariables are independent，出enthe l1-dimensional considerations are mere extrapolations of the由民e-dimensiona1reality. Of course， we cannot geometrica1ly defin巴themthough we will be working with thc numbers. The side constraints Iimit the search陀gion，implying that only solutions山atlie within a c巴rtainregion will be acceptable. They define an n-dimensional rectangular region (hypercube) from which白efeasible and optimal solutions must be chosen. Later， we wi¥l s白血atthe mathematica1 models in optimization are usually described by nonlinear relationships.τ11e solutions to such problems cannot be ana1ytically predicted鎚 theyare typica1ly govemed by由eunderlying numerical tec加 iqueused to solve them. It is neccssary to (田町ictthe solutions to an acceptable region. The side constraints provide s ready mechanism for implementing this limit. Care must be taken that thcse limits are not imposed overzealously. Thcre must be reasonable space for出巴 numericaltechniques 10 operate.

The Standard Format: The above definitions a1low us to assemble Ihe general abs回 .ctmathernatical model as

Minimize f(x.， x2， . . . ，xn)

Subject to: hl(x" X2・・・・，XII)=O

h2(x.， x2， . . . ，xn) = 0 (1.2)

、‘.，1

• l

，，‘、

h/(XI' X2・・..，XII) =0

8.(X" X2・・..，Xn)壬O

82(XI' X2，・・・，Xn)S 0 (1.3)

gm(XI・X2・・..，Xn) SO

xl SXj Sxi， i = 1，2，...， n (1.4)

The same problem can be expressed concisely using the following notation:

Minimize f(x.， X2， . . . ，Xn) (1.5)

Subject to: hk(xt， X2・・・・，Xn) = 0， k = 1， 2， . . . ， I (1.6)

め(X"X2， • . . ，XII) S 0， j = 1，2，. . . . m (1.7)

画幽画面孟副

10 INTROOUCT10N

xlsxjSxれ i=1，2，... ，n

Exploiting vector notation the mathematical model is

Minimize j(X)， [X].

Subject to: [h(X)]1 = 0

匂(X)]tnS 0

X10wSX S Xup

(1.8)

(1.9)

(1.10)

( 1.11)

(1.12)

The above mathematical model expresses恥 followingstandard format of the

optimiz岨onproblem expressed in naturallanguage:

Minimize the objective functionj， subject to 1 equa¥ity constraints， m incquality constraints， with山en design variables Iying between pr，ωcribed lower and

upper limits.

百letechniqucs in出isbook will apply to thc problem describcd in出cabovc format. To solve any specific design problem it is requircd 10 refunnulatc the problcm in the above manner so白紙自己 methodscan be applied directly. Also in this book， the techniques are developed progressively by considering the standard model wi出

向ducedelements. For example， th巴unconstrainedproblem wi¥l be explo問dfirst. The

equality cons回 intsa陀 considerednext， followed by the inequality constraints.叩 d

finally the complete model. This represents a natural progression as prior knowledge is used to develop additional conditions白紙 needto be satisfied by the solution in

白隠einstances.

1.1.2 Mathematlcal Modellng

In this section，出ethree design problems i附'Oducedearlier will be佐anslatedto出estandard format. The mathematical mode1 will first be identified. The second problem requires information from a course in mechanics and materials. This should be within 血.epurview of most engineering students.

Example 1.1 New consumer research indicates山atpeople like to drink about 0.5

liter of soda pop at a time during the summer months. The fabrication cost of the

redesigned soda can is proportional 10 the surface area and can be estimaled al $1.00

per square meter of lhe mat巴rialused. A circular cross section is the most plausible given current looling available for manufaclure. For aesthetic reasons， tl1e height must be at least twice tl1e diameter. Studies indicate血atholding comfort requires a

diameter between 6 and 9 cm.

Figurc 1.1 shows a sketch of tl1e can. In most product designs， particularly in engin巴ering，the model is easier to develop using a figure. The diameter d and the height h are sufficient to dcscribe tl1e soda can. What about the tl1ickness t of出e

，~

p J :<:

d

1.1 OPTIMIZATION FUNOAMENTAlS 11

d

h

Flgure 1.1 Example 1.1-Design 01 a new beverage can.

matcrial of the can? What arc lhe assumptions for thc dcsign problcm? One of the

assumplions could be lhat t is small cnough to be ignored in thc calculation of thc volume of soda in出ecan. Another assumption is that由ematerial 陀quiredfor lhe can is only the cylindrical surfac巴宙開.τ'hetop and botlom of thc can will be fitted with cnd caps 白atw泊 provide出emechanism by which the soda can bc pow叫.官邸isnot part of白is

dcsign problem. In the白百tattempt at dcveloping the mathematical model we could start

out by considering由equ叩 titiesidentified曲oveas des取1variables:

Design variables: d， h， t

Reviewing the stalement of the design problem， one of the parameters is the cost of material per unit area白紙isgiven as l(t per sqU8I官centimeter.τbisis idenlifiedぉ C.During the search for the optimal solution出isquantity will be hcld constant at出e

given value. Note that if this valuc changes， lhen出ecost of thc can will correspondingly change. This is what we mean by a design p釘ameter.Typically， change in parameters wi¥l cause the solution to 民間computed.

Design parameter: C

The design functions will include白ecomputation of the volume enclosed by the can and the surface area of the cylindrical section. The volume in the can isπcf2h14.ηle surface area is 1tdh. The aesthctic constraint requires出ath ~ 2d. The side cons甘aintson山 diameterare prescrib吋 in出eproblem. For complel巴nessthe side constraints

on恥 othervariables have to be prescribed by the designer. We can formally set up the optimization problem硲

Minimize f(d，h，t): C7rd1z (1.13)

SubjecllO: hM，h，t): 7rd2/1/4 -500 = 0 (1.14)

掴一一ーーーー

12 INTRODUCTION ぜ

3一戸/


Intuitively， therc is some concem with the problem as expressed by Equations (1.13ト(1.15)even though山巴 descriptionis valid. How can the value of the design variable t be estab1ished? The variation in t does not a仔'ectthe design. Changing the

value of t does not changef， hl， or gl・Hence，it cannot be a design variable. (Note: the variation in t may affect the value of c.) lf山iswerc a practical design problcm， then 出巴 canshave to be designed for impact and stacking strength. In白紙 C凶 e，t will

probably be a critical design variable. This will require sever心additionals凶 clural

constraints in the problem. It could serve as an interesting extension to this problem

for homework or project明leoptimization problem after dropping t佃 dexpressing

[d. h]ぉ [XI・X2]becomes:

ィ，

4

可

制

d

¥¥¥¥¥¥開制

L

ー吋

IV-1--

gM.h.t): 2d-h SO

6Sds9; 5s/ts20; O.OOIStsO.Ol (1.15)

Flgure 1.2 Example 1.2-Canlllever beam.

Minimize f(XI，X2): c:町内 ( 1.16)

(1.17)

gl(ズl・XU:2x1-X2S0 (1.18)

lntuitively，出elarger the value of F. the greater is山巴cαro田s鴎8-S総ecl叫lio叩na陥a叫lare可官潤=泊anecωs紅vtω.0 ha肌nd仙leit. 11 c棚a叩nb恥e蹴捌i均l砂yconcluded 山伽a飢tiげfFw悶 ad伽巴ωs叩ib巴selat its lower limit， a8 lhe mass of the beam will be direclly proportional 10由earea of croS8 8cc(Iun. No techniques are r，叫uiredf.or lhis conclusi.on. By .our definition， F is a go.od candidate as a par創neterrather白肌 asa design variable. Similarly， the larger the value of L，白eg問ateris the moment白紙Fgenerates about the cantilevered

end.百liswill require an increase in山earea .of cross-section. Therefore， L is not a good choice f.or山edesign variable either as the best choice f.or it will bc i臼 l.owerIimit. To represcnt material as a design variable. we necd to use its SlruClural propcrties in出eformulalion ofthe problem. Typically. a malerial is characlerized by its spec凶c

Subjecllo: hl(xl・Xv:ru弘/4-500=0

6 SX) S 9; 5 SX2 S 20

The problem問 presenledby Equati.ons (1.16)一(1.18)is恥 mathematicalmode¥ f.or

th巴designproblem expressed in the slandard format. For this problem no extranc.ous informali.on was n田氏dto set up出eproblem except for some geometrical relations.

Example 1.2 A cantilever beam needs 10 be designed to c紅rya point load F al the end .of the beam .of lenglh L. The cross section of the beam wiII be in the shape of the letter 1 (ぉferredto as阻 1・b回 m).百lebeam should meet prescribed failure criteria.

Thcre is also a Iimit on its deflecti.on. A beam of minimum mass is閃quired10 be

dωigncd. Figure 1.2 shows a side view .of the beam carrying the load and Figure 1.3 a typical

cr.oss s目白nofth巴be釘n.The I-shaped cross section is symmetric. The symbols d， tw. br， and tr c.orrespond to the dep出.ofthe bcam， thickness of the web， width of th巴flange，and thickness .of the flange， respectively. Thes巴quantitiesare sufficient to define出E

cr.oss section. A handbo.ok or textb.ook on strcngth of materials can aid白ed巴velopment.of th巴designfunctions. An important assumption for this problem is白紙

we will be w.orking within the elastic Iimit of thc material， where there is a Iinear relati.onship between血estress and the strain. AlI .of the variables identified in Figure

1.3 will s回 nglyaffect the soluti.on. They are design variables. The applied f.orce F will also directly affect the problem. So will its locati.on L. H.ow about由巳 material?Stecl is dcfinitely supcri.or 1.0 c.opper for the beam. Should F， L， and the material pr.operties be included as design variables?

bt I~ 'l

4・tW-生

平Flguro 1.3 Example 1.2-Cross-sectlonal shape.

14 INTRODUCTION

weight， y， its value ofthe modulus of elasticity. E. its modulus of rigidity. G. its yield

limit in tension and compr凶 sion.σyie'd.iωyield in sh閲兵 τyield.its ultimate streng白in tension.σult. and its ultimate strength in shear.τull・Optimi却 liontechniques identify diffe陀 ntva1ues of the design variables. If E is the variable used to閃 present出E

material design variable in a problem. then it is possible the solution can問quirea materia1白atdoes not exist，白atis. a va1ue of E still undiscover芭d.For a11 intents and

p凶 poses.也isis beyond the scope of the designer. Again. materia1 as a parameter makes a lot of.sense， and血eoptimization problem should be reinvestigated if F，ムor the material changes. Concluding from出isdiscussion on modeling. the following

Iist can be established for this example:

Design p釘祖1隠et胞er肱sぶ F.Lム.{y.芯ιG. σ叫Y'悶eld'τ1:yi陪州eld'σ円町u凶11'1:τ九E

Design variables: d， tw' b糾r'I今f

In the dev巴lopmentof the mathematical model， standard technical definitions紅巳

used. Much of the information can be obtained from a mechanical engineering

handbook. Th巴firstdesign function is出巴 weightof the beam. This is the product of y， L， and Ac， where Ac is出earea of cross section. The maximum slress due to bending

σ""nd can be calculated as the product of FLdI21c・whereιisthc momenl of inertia about thc ccntroid of the cross section along the axis parallel to the f1ange. The m凱 imumshear stress in出巴crosssection is expressed as FQcllc1w， wherc Qc is first moment of area about the cen位。idparallcl to thc flange. The maximum deflection (~)

of the beam wi11 be at the end of the beam calculated by the expression FL3/3Elc' Ac

of the cross section of the beam is 2brtr + Iw (d -2tf). The first momcnt of area Qc can bc calculated as 0.5bftf (d -tr) + 0.51w (0.5d -tf)2. Finally.出cmomcnt of inertiaιis

obtained from (b凶112)ー ((br-tw )(d -2tf)3/12). In山 following，for convcnience， we will continue to use Ae， Qc' andιinstcad of detai1ing their dependence on曲e

design va1ues. Associating x， with d， X2 with tw' X3 with br， and X4 with lr， so白atX=

[x，・X2・X3'x41，白eproblem in standard format is

Minimize .ftX): YLAC ( 1.19)

Subject to: g，(X): FLx，/21c-σyie'd ~ 0

g2(αx均):FQJ1.x:ぬx巧2-τ1:yie眼刷e

g3メ(X):FL3，可'/3Elιc 由 3m凪 ~O

0.0 1 ~ x， ~ 0.25; 0.00 1 ~ X2 ~ 0.05;

0.0 1 ~ X3 ~ 0.25; 0.00 1 ~ X4 ~ 0.05

(1.20)

To solve出isproblem， F must bc prescribed (10，000 N)， L must be givcn (3 m).

Matcria1 must bc selected (stee1 :γ= 7860 kglm3; O"yield = 250 E+06 N/m2;τyicld = 145 E+06 N/m2). A1so the maximum dct1ection is prcscribcd (amn~ = 0.005 m).

町長良民ルお

hFdf託彰もm伊佐山↑JdJiJd町山

hhおbMUFF一正訴訟記訟や払

1.1 OPTIMIZA TION FUNDAMENTALS 15

This is an example with four design variables， three inequality constraints. and no equality constraints. Othcr vcrsions of this problem can easily be formulated.

It can be reduced to a two-variablc problcm. Standard failure criteria with respect

to combined stresscs or principa1 strcsses can bc formulated. If the cantilevered end is bolted， then additiona1 design functions regarding bolt failure need to be

exar凶ned.

Example 1.3 MyPC Company has decided to invest $12 million in acquiring severa1 ncw Component P1acement Machines to manufacture di仔erentkinds of mo出eぬoards

for a new generation of personal computers. Three models of these machines are under consideration. Tota1 number of operators available is 100 becausc of出elocal labor market. A floor space constraint needs to be satisfied because of血edifferent

dimensions of恥 semachines. Additiona1 information relating to each of白e

machines is given in Table 1.1.百 ecompany wishes to determine how m叩 yofeach

kind is appropriate to maximizc the number of boards manufactu悶dperday.

lt is difficu1t to use a figure in this problem to sct up our mathematical model. The number of machines of cach model necds to be determined. This will serve as our

design variables. Let XI represent th巴numberof Component Placement Machines of

Model A. Similarly， X2 wi1l be associatcd with Model B， and X3 with Model C. i!i

D凶 igllvariubles: x，・X2，X3The infonnation in Table 1.1 is used to set up the design functions in terms of出e

design variables. An assumplion is made that all machines are run for three sh出s.The

cost of acquisition of the machines is the sum of出ecost per machine mu1tiplied by

出巴numberof machines (g 1)' Th巴machinesmust satisfy山efloor spacc constraint (82) which is defined without details in Eq. (1.23). The constraint on the number of opemtors is three timcs白巴 sumof曲eproduct of the number of machines of伺 ch

model and the operators per shift (g3)' The utilization of each machine is the number of boards per hour times the number of hours the machine operates per day. Thc optimization problem can be assembled in山巴 followingform:

ヲィf

s

Ma瓦imize f(X):18 x55xxl+18x50x~+2Ix50xx3 (1.21)

or

T Minimize f(X):-18 x 55 x x， -18 x 50 X x2 -21 X 50 xX3

Suゆctto: g，(X): 400，OOOx， + 600.000X2 + 700，000X3 ~ 12，0肌 000 (1.22)

g2(X): 3x， -X2 + X3 ~ 30 (1.23)

83(X): 3xI + 6X2 + 6x3 ~ 100 (1.24)

γ

XI ~ 0; X2 ~ 0; X) ~ 0

Eri-522JJyd罫おなおミ

一阻止一-61'甲山岨臨調ø.1.Y:.iK.~甘通

17

9

OPTrMIZATION FUNDAMENTALS

6

1.1

5

INTRODUCTION

Equations (1.21)ー(1.24)express恥 mathematicalmodel of the problem. Note伽 tinthis problem there is no product bcing dcsigned. Here a strategy for placing order for 恥 numberof machines is being dctermined. Equation (1.21) illus国防thetranslation of the m蹴 imizationobjective into an叫uivalentminimizing one. The inequality constraints in Equations (1.22)ー(1.24)are different from the previous two ex剖nples.He隠 theright-hand side of the S operator is nonzero. Similarly， the side constraints are bound on the lower side only. This is done deliberately. There is a significanl difference between出isproblem and the previous two. All of the design functions here are linear. This problem is c1assified as a linear prograrnming problem. The solutions 10 these typcs of problems are very different than thc solution 10 thc first two cxampl巴s，which are recognized as nonlin巴arprogramming probl巴ms.

1.1.3 Nature of Solution

百le出向ecxarnpI巴sin出cprevious section arc used to discuss出enature of solutionsω the optimization problem. Exarnples 1.1 and 1.2 dcscribe血enonlin田rprogramming problcm， which forms the major part of山isbook. Exarnple 1.3 is a lincar programming problem， which is very significant in dccision scicnccs， but ra路 inproduct design. Its inc1usion here， while necessary for complctcness， is also important for understanding contemporary optimization techniques for nonlinear programming problcms. Exarnples 1.1 and 1.2a回 dealtwith ftrSt. Between the lwo， we notice that Exarnple 1.1 is quite simple relative to Exarnple 1.2. Second， the four variables in Example 1.2 make it difficult to use i1lustration to establish some of thc discussion.

16

G~叩hl伺I representation 01 Example 1.1.

bo山 ofthesc functions.百le叫ualityconstraint gl is lincar. ln Equation (1.18)出isis evidentb∞ause the desi伊 variablesappcar by出emselveswithout being raised 10 a powcr o山町出回1.As we develop白etechniques for applying optimization，出isdistinction is important to keep in mind. It is also significant白紙 graphicalrepresentation is typically restricted to two variables. For出reevariables， we need a fourth dimension 10 resolve the informalIon and three-dimensional conlour plots are not easy to iIIustrate. The power of imagination is necessary to overcome自己sehurdles.

百1巴problemreprcsented in Figure 1.5 providcs叩 opportunityto identify the graphical solution. First， the feasible region is identified. In Figure 1.5山isis出E叫ualitycons回 intabove the incquality cons回int.Bothfeωible and oplimal must occur from this 陀:gion.Any solution from出isregion is an acceplable (仰 sible)solution. There are usually a large number of such solutionsー infinilesolutions. If optimization is involved， thcn thesc choices must be rcduced to the best one with陀 spcctto some criteriaー theobjeclive. 1n this problemlfigure， the smallcst value offis desired. The lowest valuc of f is just less than the contour value of 308. lt is at山eintersection of the two constraints. While the value of f needs to be calculated， the optimal values of the design variables， read from the figure， are about6.75 and 13.5， respectively. Another significant item of information obtained from Figure 1.5 is白紙 glis an active constraint. While there are infinite feasible solutions 10 thc problem， the optimal solution is Ultique.

Flgure 1.4

ん量まー与一式配EZ白羽

[~

5

SOLUT10N TO EXAMPLE 1.1 Thc simplcst determination of nonlinearity is through a graphical 陀P陀 sentationof出cdcsign functions involved in thc problem. 百liscan be donc回 silyfor ODe or two variables. If the fUDCtiOD does not plot as a straight line or a plane，出enit is nonlincar. Figu隠 1.4shows a three-dimcnsional plot of Examplc 1.1. The figure is obtained using MATLAB. Chapter 2 will provid巴dctailcdinstructions for drawing such plots for graphical optimization. The threc-dimcnsional 問pr凶 entationof Exarnple 1.1 in Figure 1.4 does not real1y enhance our understanding of the problem or the solution. Figure 1.5 is an altemate rep陀scntationof the problem. This is th巴oneencouraged in this book. Thc horizontal蹴 isrep陀 scntsthe diameter (XI)， and出巴 verticalaxis由eheight (xu. 1n Figure 1.5 the equali砂constraintis markcd appropriately as h 1. Any p創rof valucs on出isline will give a volume of 500 cm3• The inequa/ity constraint is shown as g 1. The pair of values on this line exact1y satisfies thc aes山eticrequi陀 ment.Hash lines on the side of the inequality constraint cstablish the disallowed陀 gionfor the design variables.百leconstraints are drawn thicker for emph凶 is.The scaled 0匂'ecliveルnClionis repr，ωen句dthrough sevcral labelcd contours. Each contour is associated with a白xedvalue of the objcctive function and these values are shown on the figure.百lerange of the two axes establishes the side COllStraints. The objective functionf and the equalily constraint h 1

創'cnonlinear sincc they do not plot IIS straight lines. Rcferring to Equations (1.16) and (1.17)， this is substantiated by the pl'OduCIS of the two unknowns (design varillbles) in

19

An interesting observation in the above set of equations is山atthere are only

two design variables but three constraints. This is a valid problem. If the constraints were equality constraints，出enthis would not be an acceptable problem definition as it would violate白erelationship between the number of

variables and the number of equality constraints. Therefore， the number of inequaliηconstraints are not related to the numb巴rof design variables used. This fact also applies to nonlinear constrnints. Figure 1.6 is a scaled plot ofゆcfunctions (1.25)ー(1.28).Choosing the first quadrant satisfies the side constraints.

τ'he linearity of all of the functions is indicated by the straight-line plots on the

figure. Again， several contours offare shown to aid the recognition ofthe solution. All of the inequalities are drawn with hash marks indicating the nonfeasib1e side

of the constraint. The feasible region is enclosed by the two axes along with

constraints g2 and g3' The objective is to increas巴thevalue off(which is indicated by the contour levels) without leaving the feasible region. The solution can be

(1.28)

1.1 OPTIMIZATION FUNOAMENTALS

83(X): 3xI + 6x2 ~ 70

X2~0 X1 ~O;

Jtub//drιvdrfLbトff/yJX

INTROOUCTION 18

20

{EHU}-Z司

34

25 20

Conlour plol 01 modllled Example 1.3.

1S number 01 Model A

10

Flgure 1.6

5

9

SOLUTION TO EXAMPLE 1.3 In contrast to Example 1.1， all of the relationships in恥 mathematicalmodel， expressed by Equations (1.21)ー(1.24)，are linear. The

word equation is generally used to describe the equivalence of two quantities on either side of the = sign. In optirnization， we come across mostly inequalities.百】eword

equatioll will be出 edto include both of these situations. In Example 1.1，出ena随時

of solution was explained using Figu問1.5.In Example 1.3， the p閃senceof thr田

design variables denies山isapproach. Instead of designing a new problem， suppose 白紙 thecompany p毘 sidenthas d配 idedto buy five machines of the third type. The

mathematical model， with two design variables Xl and X2' is reconstructed using the information that X3 is now a parameter with the value of 5. Substituting for x3 in the

model for Example 1.3， and cleaning up the first cons回 int:

M砿 imize

Subject to:

(1.25)

(1.26)

( 1.27)

H137 8.5

j(X): 990 XXI + 9∞xX2+5250

自

Conlour plol 01 Example 1.1.

glぽ):O.4xl + 0.6x2 ~ 8.5

7.5 dlameler (cm)

g2(X): 3xI -X2 ~ 25

7

Agure 1.5

6.5 5 6

20 lNTRODUCT10N

spottcd at the intersection of constraints g2叩 dg3. The values of the design vari油Ics

at the solution can be read off from thc plot. It is not over yet. Before proceeding

further il is neccssary to acknowledgc that a solution of 6.5 machines for Model B

is unacceptable. The solution must be adjusted to the ncarest integer values. In

aClual practice. this adjustmcnt has to bc made without violating any of the

constraints. The need for integer variables is an important consideration in dcsign.

V回ables白atar巴飽quiredto have only integer values belong to白etype of variables

called discrete variables.τ'his book most1y considers continuous variables. Almost a11

of the mathematics an engin田 rencounters， especially academically， belong to曲E

domain of continuous variables. Thc assumption of continuity providcs very fast and

cfficicnt techniques for the search of constrained optimum solutions， as we will develop in this text. Many real-Iifc designs， especially those that use off-the-shelf materials and components， determine discrete models. Discrete programming and integer programming are disciplines that address these models. The nature of出巴

search methods，出atis， methods used for searching出eoptimal solution， in discrctc

prograrnming is very different from山atin continuous programming. Many of th凶巴

methods are based on the scanning and replacement of the solutions through

exhauslive search of the design space in the design region. In this book， disc陀tc

problems arc typically handled as if they were continuous problems. The conversion

to由ediscrcte solution is the final part of the design effort and is usually performed

by the designer outsid巴 ofany optimization technique we develop in this book.

Chapter 8 discusses discrete optimization with examples.

Getting back to出elinear programming problem just discussed， it should also be noted that only the boundary of the feasible region will affect the solution. He陀 isa

way that will make白isapparcnt. Note that the obj巴ctivefunction contours ar官straight

lines. A different objective function will be displayed by parallellines with a different

slope. Imagine these Iines on the figure instead of the actllal objective. Note出atthe

solution always appears to lie at th巴intcrsectionof the constraints， that is， th巴cornersofthed巴signspacc. Solution can nevcr com巴frominside of the fcasible 陀 gion，unlike in the nonlinear programming problem， which accommodates solution on the boundary and from within the feasible region. Recognizing thc natu問 ofsolutions is

important in developing a search procedure. For example， in linear prograrnming problems， a technique employcd to search for the optimal d巴signneed only search thc

boundaries， particularly the intcrsection of the boundaries， whereas泊 nonlinear

problems， the search procedure cannot ignore interior values.

1.1.4 Characterlstlcs of the Search Procedure

百lesearch for the optimal solution will depend on出enature of the problem being

solved. For nonlinear problems， it must consider the fact that solution could lie inside

the feasible region. Example 1.1 throllgh Figurc 1.5 will provide出econtext for

discussing the scarch procedure for nonlincar problems. Simi1arIy， the characteristics

1.1 OPT1M1ZAT10N FUNDAMENTALS 21

of the search process for linear problcms will be explored using Ex四 lple1.3 and

Figure 1.6.

山

pd引γ一一yd吃

dydpdd尚一一位三huuk一一xydyM同号drぉ

i

Nonllnear Prob/ems: Except for a few c1asses of problcms， solutions to non1inear problems are obtained through numerical analysis. Through computβr code， numerica1 analysis becomes numcrical techniques. The methods or techniques for

finding the solution to optimization prob1ems are called search methods. In applied

mathematics and numerical t田 hniquesthese創買う referredto as iterative techniques.

Implicit1y this means seveml tries will be necessary beforc the solution can be

obtained. A1so implied is that cach try or search is executed in a consistent manner.

Information from山 previousiteration is utilized in the computation of values in the

present sequence. This consistcnt manner or pr∞ess by which the search is carried out

and恥 solutiondetermined is ca¥1ed叩 algorithm.This term a1so refers to the tr佃 slationof the particular search procedure into an ordered s叫uenceof step-by・step

actions. The advantage is derived mostly in the conversion of these steps， through some computer 1anguage， into code出atwi1l execute on山ecomputcr. Developing山E

a1gorithm and trans1ating it into code that wil¥ execute in MATLAB wil¥ be a standard

approach in白isbook. The search process is started typica¥1y by trying to guess th巴initialdesign solution.

The designer can base this selection on his experience. On sevcm1 occasions the success of optimization may hinge on his ability to choose a good initial solution. The

search method， even when used consistent1y， responds different1y to the particular

problem出atis being solved. The degree and type of nonlinearity may frequent1y

cause白emethod to fail， assurning there are no inconsistencies in problem

formu1ation. Often，出巴numberof design variables，出enumber of cons回 ints，as we¥1

as a bad initial guess p1ay a part in unexpected behavior under standard

imDlementation of the method. This has led to the creation of seveml differcnt

techniques for optimization. Practica1 experience bas a1so suggested白ata class of

problems respond well to certain a1gorithms. The mcthods for出esearch， or it巴mtivemethods. are current1y the only way to

implement numerical solutions to nonlinear problems， irrespective of the discipline in which these problems app回 r.百1凶 eproblems are p1entifu1 in design optimization.

optimal contro1. structural and t1uid mechanics. quantum physics. and astronomy. Whi1e optimization techniques are applied回allof the noted disciplines. in回 ditional

design

予

的

23

at a distan四 sS..This point satisfies the constraint h.. A出irds岡崎Ywould be to choosc X.，・Thislies on the constraint boundary of g.. lt is to be noted that all出向E

choices are not feasible. Which point is chosen is dccided by the rules of a particular algorithm. Assume point X) is the next point in出isdiscussion.

Point X) now becomes恥 newpoint xo・Anew剛氏hdircction ~ is de匂nnin回.

Point X2 is 10伺旬d.Anothぽ S明uenceof calculations locat郎 X3• 百lenext iteration yields 白esolution. The essence of updating曲edesi伊 duringsuc偲 ssiveiteration lies in two major computations， thc sea陀 hdirectioll 8/ and stepsizeα.百lCchange in由edesign vector， defin氏Ias .6X， which represents出巴 veclordiffcrence， for ex創nple，X2 -X.， is obtained asαS/. For each iteration the following s珂uenceof activity can be identified:

The above is the basic structure of a typical optimization algorithm. A complete one will indicate出巴 m加 nerin which S and αare computed. Il wil1 incJude tests， which will determinc if山巴 solutionhas bcen reached (convergence)， or if the procedure needs to be suspended or restarted. The diffcrent techniques to b巴explorcdin this book mostly differ in出eway S is established. Take a look at Figure 1.7 again. At Xo there are infinite choices for the search direction. To reach the solution in 民邸onabletime an acceptable algorithm will try not to search along directions白紙donot improve the current design.百lereare many ways to accomplish出isstrategy.

1.1 OPTIMIZATION FUNDAMENTALS

ChooseXo Identify S Determineα AX=αS XMW= Xo+.6X SetXo ←Xnen GoToStepO

8tep 0: Step 1:

:r;-:.

INTRODUCTION

FJgure 1.7

g 8.5 日

A typical ssarch method.

7.5 dlamoler (cm)

22

20

5 6

{ESEmgz

Linear Programming Problem: It was previously mentioned白紙thesolution to 白巴seproblems would be on the boundary of白巴 feasible陀gioo(also called the j面倒ibledomain). In Figure 1.6，出e蹴 esand some of the cons回 intsdetennine this region. The lalter is distinguished also by出epoints of interscction of the constrainLs among themselves，出eorigin， and白eintersection of the binding (active) coostraints with the axes. These are vertices or corners of the quadrilateral. Essentially the design improves by moving from one of these vertices to the next one through improving feasible designs. To do that you have to start at a feasibl巴comer.This will be explored more fully in the chapter on linear programming. It is to be emphasized出atonlywith two variables can we actually see the geometric iIIustration of the technique，白紙 is，the selection of vertices as the solution converges.

This part of Chapter 1 has dealt with the introduction of the op白泌却tionproblem. While it was approached from an engineering design perspcctive， the mathematical model， in abstract terms， is not sensitive to any p町ticulardiscipline. A few design rules with resp回 tto specific examples were examined. TIle standard mathematical model for optimization problems was established. Two broad cJasses of problems，

initial starting伊 essis白epoint marked xo・Remember白紙 whensolving山

optimization problem. such a figure does not目 ist.If it did，血enwe can easily obtain the solution by inspection.百lisstatement is espccially true if出ereare more than two variables. In the event th巴陀 aretwo variables，自己防 isno reason not to obtain a graphical solution (Chapter 2). Thc mcthods in optimization rely on thc dircct ex凶 polationof the concepts and ideas develo戸dfor two variables to any number of design variables. That is the reason we will use two variables to discuss and design most of the algorithms and methods. Moreover， using two variables，自己conceptscan also be reinfor四 dusing geometry. The design at the point Xo is not feasible. lt is not useful to discuss whether it is an optimal solution. At least one more iteration is necessary to ob凶nthe solution. Consider出enext solution at XI. Most methods in optimization will attempt to move to X I by first identifying a search direction， S 1・Thisis a vector (same dimension as design variable vcctor X) poi凶略的mXoloXI・百eactual distance from xo to XI・callit XoX.. is some multiple of SI' s町 αSI・X10nceagain is not feasible altbough it is an improvement on xo since it satis日目白E

incqualily constrainl gl (lying in lhe inlerior of出econstraint). Altemately， using lhe same scal'ch dircction S 1， another techniqueJme出odmay choose for its next point， X 1'，

…一一一.1""':-......，，1:一一一一 n' …可'一ー

24 INTRODUCTION

linear and nonlinear. were introduc凶.Thegeome汀icalcharacteristics of the solution for thesc problems were shown血rough白巴graphicaldescription of出eproblems. An overview of出esearch techniques for obtaining the solution to these problems wω also iIIustrated. It was a1so mentioned出at白esolutions were to be obtained numerical，か.primarily through digital computation.

The numerical methods used in山isarea are iterative methods. Numerical solutions involve two components:白附，叩 algorithmthat will巴stablish出巴 iterativeset of calculations. and second. the translation of the algorithm into computer codes. using aprogramming lang凶 'ge.This finished code is referred to箇 software.Until recently in engineering education as well as practice. the compu旬rlanguage of choice was predominantly FORTRAN with a Iittle Pascal and C. The computational environment in which it was delivered and practiced wお山emaiψ'amecomputing systems. Today，

由escene is very different. Individual personal computers伊Cs)，running Windows， Macintosh， LINUX， or other flavors of UNIX， operating systems are now the means for technical computing. They possess more power than the old main合amecomputers and they are inexpensive. The power of the graphical user interface (GUI)，出eway 出euser interacts wi白 thecomputer and山esoftware， hぉ revolutionizedthe creation of software today. New programming paradigms，抑制a11y0匂'ecloriented

programming， have led to new and improved ways to develop software. Today， cnginccring softwarc is crcatcd by Buch gcncl'al programming languagcs IlS C， C++， Java， Pascal， and Visual Basic. Many software vendors develop their own extensiol1s to the standard language， providing a comple1e巴nvironmentfor software dev巴lopmentby出euser. This makes it very convenient for users who are not necessarily programmers to get白eirjob done in a reasonab1y short time.

One such vendor is Ma出Wo此s，wi山 theirflagship product MA TLAB a10ng with a complementary collection of toolboxes出atdeliver specific discipline-oriented information. In血isb∞k，MATLAB is used exclusively for numerical support. No prior familiarity with MATLAB is expected， though familiarity with the PC is required. Like all higher-1cvcllanguages， effectiveness with this new programming resource comes with effort， frequency of use. original work， and a consistent manner of application. By the cnd of this book， it is hoped the問 aderwill be very familiar with thc u

京

1.2 INTRODUCTION TO MA11.AII 25

1.2 INTRODUCTlON TO MATLAB

JAγ22一円//一

J1引けす

MATLAB is introduccd by MathWorks Inc.， as the language for technical computing. Borrowing from the description in叩 yone of its manuals， MATLAB integrates ∞mputation， visualizatio民間dprogramming in an easy-to・useenvironment where problems and solulions are expressed in farniliar mathematical notation. The typical

uses for MATI..AD includ巴

• Math and computation

・A1goritllmdevelopment

• Modeling， simulation， and prototyping

• Data analysis， exploration. and visualization ・Scientificand engineering graphics

• Application development， including GUI building

The use ofMATI..AB has exploded recently and so have iωfeatures. Tbis continuous enhancement makes MATI..AB an ideal vehicle for exploring problems in all disciplines that manipu1ate mathematical content. This abi1ity is multiplied by application-specific solutions through toolboxes. To learn more about the latest MATI..AB. its features， the toolboxes， collection of user-supported archives， information about Usenet activities， other forums， information about books， and 80 on， visit MATLAIl a1

hup:/lwww.ma1hworks.com.

1.2.1 Why MA11.AB?

MATLAB is a standard tool for in佐'oductoryand advanced courses in mathematics， engineering， and science in many universities around曲巴world.In industry， it is a too1 of choice for res悶悶h，development， and analysis. In山isbook， MATLAB'S basic array element is exploited to manipulate vectors and matrices that are natural to numerical techniques. In the next chapter， its powerful visualization features are used for graphical optimization. In出erest of this book， the interactive featurl凶佃dbuilt-in support of MATLAB are utilized to translate a1gorithms into functioning code， in a fraction of the time needed in other languages like C or FORTRAN. Toward白eend，we will in町oducethe use of its Optimi回 tionToolbox.

Most books on optimization provide excell巴ntcoverage on the ~回hniques and a1gorithms， and some a1so provide a printed version of the code. The code is民qui陀 dto be used in mainfmmEor legacy environments that must support a compiler in the language出ecode is written， which is predominantly FORTRAN. Each student出en釘anslatesthe orinted code into a forrn suitable for computation.百lenext step is to compile the code， debug compilation errors， run出巴 program，and troublωhoot execution crrors before useful隠sultscan be obtained. An important consideration is 白紙 thestudent is expected to be conversant wi由 theprogramming language. While FORTRAN was formally taught to engin田 rsseveral y回rsago，自isis no longer true.

』竺7十竺一一ー -一一一一一一』

26 INTRODUCTION

Many electrical engineering departments have started to embrace C and C++. PCs are great for the use of ready-made applications， especially those with GUI， but its use for programming in FORTRAN has yet to take off.

MATI..AB suffers very few of these disadvantages. Its ∞de is compact釦 ditisinωitive to leam. To make problems work requires few lines of code compared to traditional languages. Instead of programming from白eground up， stan伽rdMA TI..AB pieces of code canbe由民adedtoge出erωimplementthe new algorithm. It avoids the s包ndard鉛 :pru:ヨ飽∞mpilation and link s叫uenceby using叩 inte.叩reter.Another important隠 asoniscode po巾 bility.官le∞dewritten for出ePC version ofMATI.AB d田 snot have to be changed for the Macintosh or the UNIX systcms.百liswill not be true if system-dependent resources mωedin白ec吋e.As an ilIustration of MATI.AB'S leaming agility， bo出 MATI.AB叩dapplied optimization are∞vered in this book， something very diffic凶t旬 a∞omplishwi出，say， FORTRAN and optimization， or C++叩 doptimization. Fl町出ermore，恥IA'百..AB，like C or FORTRAN， is a globally relevant product.

1.2.2 MATLAB Installalion Issues

This book does not Care if you are running MATLAB on an individual PC or Mac， a networked PC or Mac， or individual or networked Unix workstatioJls. MATLAB takcs care of this issu巴.The book assumes you have a working and appropriately Iicensed copy of MATLAB in the machine in合ontof you. On your personal machine， for installing MATI.AB， follow the instructions that accompani巴dthe box. Typically白esystem administrator handles network installations.

This book uses MATt.AB Version 5.2 (until Chapter 4) and Version 5.3 (山erest). Version 6 has been shipping since the fall of 2000. MA礼 ABversion 5.0 and above should work. If MATI.AB has changed any built-in commands in newer versions血佃

the one凶 edin the book， it should provide information on the availability of newer version of山 commands(or lhrow an error， a rare case). The MATLAB code and examples were created on an Inlel Pentium Pro 200 running the Windows NT 4 operating syslem. The earlier codes were also tested on a DEC Alpha server running OSFl V4. The MATI..AB version on the server was 5.1.

In organizations and instilutions MATt.AB software is usually licensed on a subscription basis. Users therefore hav巴the1atest version of the software in these situations. In such a case the new version (at出巴timethe book is first published)， will be Version 6 (or Release R 12). This book is based on Version 5 (Release RI1).百lecode胡 d由巳 programminginstructions in the book shou1d execute withoUl problem in the new version. In the new Release R12， on the PC p1atform， the default appearance of MATI..AB is a composite of several windows presenting different pieces of information. This is shown in Figure 1.8. This c回 becustomized through the menu b釘 byView →Desktop Layout→Command Window Only， which also coπωponds to山eprevious問 1ease.It is shown in Figure 1.9.百lisview is simple and less distracting for getting familiar wi出MATl..AB.Note白紙白eMATt.AB command prompt is >>. This window delivers standard operating system commands for saving， printing， opening files， and so on， lhrough menus and a command bar. In order to

T信配

h' m.

f.

1.2 INTRODucnON TO MATl.AB 27

F1gure 1.8 MATLAB opening view.

create files (or code) another graphica1 window is n巴eded.This is山eMATLABM・fileeditor/debugger window and is shown in Figu偲1.10.The editor uses co10r for enhancing code readability. Each of t山hes弱ewindows spawn additiona1 dia10g boxes (if

n附e回c側町均)fi伽oωぽr日川.s民刷削則c“削ttωti同n略1唱gfi蜘e伺側a瓜tu附re附e邸sd削u町叫Iげ川ri叫irworkspace browser.

While this is not so much as an installation issue， it wou1d be appropriale lo discuss山eway仙胸kplaI1s to wri包 andexec雌 MATI.ABC峨 Gen叫 y，出ereare two ways to work明白MATI.AB:interactivelyand throゆ附'pts.In interactive mode.自己∞rnn1andsareente陀d田 d郎氏utedone at a time in the MATt.AB cOrnn1and window. Once出巴sess10nis over and MATt.AB exited， there is no information 1也tainedon尚早set of commands白紙

were used and what worked (unless a choice was made to save the session). Large s詑ea叫ue叩n隠 sof cOrnn1aI1ds are usually di血cu凶1uω.ok伐 P釘回ac依kof泊加 i加n配E低臨erac凶Uv刊巴 se白ss劉10n.

入ぷ…4ゐo伽 o∞叫p飢凶t首帥i白印畠filein MATI.AB・Thisis called a script m-ftle. Most MATLAB fil郎町田lledm-ftles. The m is a meextBIESion usually mserved for MA-I-LAB.Any code changEsmemade thmush 山eeditor. the changes to 出efile saved， and 山e filcexecu飽din MATt.AB again. Ex巴伐cu凶』胤tin昭

M肱ATt.A凶叫Btl剛 ghs氏cふ勾叩d均印p似川州t凶巾白m伽le加eω削帥s叫山iお切s叩pr叫e吋do叩側m削1泊lin

tωore芭verttωo interactive mode any time.

1.2.3 Using MATLAB the First Time

E喝g

MATωwi¥l be used inte削 ive1yin出issection. The au.thor very叩n斜yrecornn1ends a hands-on approach to understanding MATt.AB・Thismeans typmg tt1e codes yourself. It a1so inc1udes underslanding the syntax巴rrors，and debugging and

28 INTRODUCTION

百 isis a Classtoom License fOt insttuctional use only. Reseatch and com且etcialuse is ptohib1ted.

To qet statted， select "HATLAB Help" ftom .the Help menu.

Flgure 1.9 MATLAB Command wlndow.

Optim1zation官1ほ1~IATtAB

'iI John ~iley も Dt.P. VenkataJ;amenl

F1gure 1.10 MaUab Editor/Debugger window.

1.2 INTRODUCTION TO MAlUB 29

rectiちring白ecode. The author is not aware of a single example where programming

was learned without the reader typing叩 de.ln出isb∞k， MATLAB code segments are courier font with出ebold style lIsed for emphasizing commands or other pieces of

information. Anything else is recommendations， suggestions， or exercises. The

placeholder for command is in italic.

さl

Before We Start: The fo¥lowing punctuation卸 dspecial characters are worth

noting:

者 A¥l text after the % sign is considered a

comment. MATLAB ignores anything to

the right of the % sign. Comments are used liberally in this book to indicate

why's and what's of command or code

usage. Do not overlook them. You do not

have to code them in， however， probably two weeks later you wi¥l not remember

why you developed that particular piece

of code.

This is the default MATLAB prompt. A semicolon at the end of a line prevents

MA礼 ABfrom echoing the information you

enter on the screen.

A comma will cause the information to

A 叶

計

，. J

¥h. U

>>

住宅志野

Ft主主島知J-

一

echo. Default is a comma.

A succession of three periods at the end of

the line informs MATLAB白紙 codewill

continue ωthe next line. You cannot split

a variable name across two lines.You

cannot continue a comment on another

Iin巴.You can s試to叩'pMATLAB execution and get

back t出h巴c∞omm!削mdpromp戸tb句ytyping c

(C佐凶l-C)一b句yh加01凶din昭gdown

tωog酔e出eぽr.

WilI display information on the various

ways the command can be used. This is the

quickest way to use help.

Opens a help text window that provides

more information on all of the MATLAB

resources installed on your system.

Provides help using a browser window.

vl院主rEYHHιEJ民rpL耐震主訟叫世主ドi

A

c

>> help command_name

>> helpwin

>> helpdesk

30 INTAODUCTION

For those of you programming for the first time， the equal to sign (=) has a very special

meaning in all programming languages， incJuding MATLAB. It is called the assignment operalor.百levariable on thc Icft-hand side of the sign is assigned the value of山e

right-hand side. Thc actual eqllallO operation in MATLAB is usually accomplished by

a double cqual to sign (==).

Some Additionsl Features

• MA礼 ABis case sensitive. An a is different than A. This is difficult for persons who arc uscd to FORTRAN. All built-in MATLAB commands arc in

lowercase.

• MATLAB does not necd a type definition or a dimension statement to introduce variables. It automatically cr四 tesone on first encounter in code. The type is

assigned in context

• Variabl巴namesstart with a letter and contain up to 31 cbaractcrs (only Ictt巴rs，digits， and underscorc).

• MATLAB uscs some built-in variable names. Avoid using built-in variable namω.

• All numbers arc storcd internally using the long format spccificd by IEEE

f1oating-point standard. These numbers have roughly a precision of 16 decimal digits. They range roughly between 10E-308 and 10E+308. Howeve丸山巴yare

displayed differcntly depending on the context.

• MATLAB uses conventional decimal notation for numbers using decimal

points and leading signs optionally， for example， 1， -9， +9.0， 0.001，

99.9999.

• Scientific notation is cxprcssed with the letter e， for cxample， 2.0e-03， 1.07e23，

ー1.732e+03.

• lmaginary numbers use either i or jωa suffix， for example， li， -3.l4j， 3e5i.

Operators: The following arc the arithmetic operators in MATLAB.

+ Addition (wben adding matriccs/arrays size must match)

Sub釘action(same as above)

* Multiplication (the subscripts of arrays must be consistent when multiplying them)

/ Division

Power

Complcx co吋ugat巴transpose(山0町 aytranspose)

In the case of arrays， each of these operators can be used wi白aperiod prefixed to白e

operator， for cxample， ( . * ) or ( • ) or (1).明白hasa special meaning in MATLAB. It

imp1ies巴lement-by-elementoperation. It is uscful for quick computation. It has no

五"

1.2 INTAODUCTlON TO MATlAD 31

relevance in mathematics or anywhere else. We wiJ1 usc it a lot in the next chapter for gClIerating data for graphical op出凶zation.

An Interactlve Sesslon: S也氏 MA礼 AB.On恥 PCprcss the Start button. Press the Program button. Prcss thc MATLAB 針。'gramGroup. Press MA TLAB. The MATLAB Command window opens up and after somc introductory messages白E悶 willbe a MATLAB promptindi回悦dby double forward arrow.百lisis叩 indication白紙MATLABis ready to aαeptyour∞mmands. On the networked s也tions，typing matJab at the window prompt should do the same. Note that the MATLAB program must be in the pa出asdefmed in出e.Iogin or.ωh代 file.Ask the lab assistant for help. Thωe紅巳standardUNIX proccd町民・

凶t陀知rnat the end of出elinc or befo問山ecomment. The commcnts rclatc to what is being typed加 dwhat is displayed in the MATLAB command window. FI田 1frccω町 yourown variatiosS. In fact， to unde岡田dandreinfo詑巴白巴commands，it is 問commended白紙

you make up your own examples often.

>>a 1.0; b 2.0; c 3.0， d 4.0; e 5.0 >>苦 whydid only c and e echo on the screen? >> who 者 listsall the variables in the workspace >> a 者 givesthe value stored in a >>A 1.5; 告 variableA >> a， A % case matters >>one a; two b; three C; >>苦 assigningvalues to new variables >>four d; five e; six pi;者 valueof pi available >> A1 [a b c ;' d e f] 者 A1is a 2 by 3 matrix >> % space or comma separates columns >> % semi-colon separates rows

>> A1(2，2) 者 accessesthe Matrix element on the >> % second row and second column >> size (A1l も givesyou the size of the matrix >>者 (row，columIlsl >>AA1 size(A1l 者 Whatshould happen here? >>者 fromprevious sヒatementthe size of A1 >>も containstwo numbers organized as a row >>告 matrix. This is assigned to AA1 >> size(AA1l も AA1is a one by two matrix >> A1' 者 thistransposes the matrix A1 >>B1 A1' も thetranspose of matrix A1 >>者 isassigned to B1. B1 is a three by two matrix

>> C1 '" A1 * B1者 SinceA1 and B1 are matrices this >>者 i8a matrix multiplication > > 巷 Shouldthis multiplication be allowed?

>>も considermatrix multiplication in C or

32 INTRODUCTION

>> % Fortran >>者 thepower of MATLAB derives from its >>も abilityto handle matrices like numbers >>C2 81 * A1も Howabout this? >> C1 * C2 も Whatabout this? >> % read the error message >>者 itis quite informative >>D1 [1 2]' 苦 D1is a column vector >>C3 [C1 D1] 者 C1is augmented by an extra column >>C3 [C3 C2(3， :)]告 Note is an assignment >>告 meansdo the right hand side and overwrite the > >告 oldinformation in C3 with the result > >告 ofthe right >>告 handside calculation

>>告 Onthe right you are adding a row to current >>者 matrixC3. This row has the value of the third >>告 rowof C2 -Notice the procedure of >>奄 identifyingthe third row. The colon >> % represents all the columns >>C4 C2 * C3 も permissiblemultiplication >>告 Notethe presence of a scaling factor >>者 inthe displayed output >>CS C2 * C3 告 seemsto multiplyl >>告 Isthere a difference between C4 and CS? >> % The * represents the product of each element >>者 ofC2 multiplied with the corresponding >>者 elementof C3

>>C6 inverse(C2) % find the inverse of C2 >>告 apparentlyinverse is not a command in MATLAB >>者 ifcommand name is known it is easy to obヒain>>者 help

>> lookfor inverse告 thiscommand will find all files >>者 whereit comes across the word "inverse" in >>者 theinitial comment lines >>者 Thecommand we need appears to be INV which > >告 saysInverse of a Matrix >> % The actual command is in lower case. To find >>告 outhow to use it -Now >> help inv % shows how to use the command >> inv(C2) 者 inverseof C2 >> for i '" 1:20

f(i) i"'2; end

> >告 Thisis an example of a for loop

1.2 INTRODUCnON TO MAn.AD 33

>>者 theindex ranges from 1 to 20 in steps of >>者 1(default) >>者 theloop is terminated with "end" >>者 theprompt does not appear until "end" is >>者 entered>> plot(sin(0.01*f)，cos(0.03合 f)) >> xlabel('sin(O.Ol*f)') 者 stringsappear in single >>毛 quotes>> ylabel('cos(0.03*f)') >> legend ('Example') >> title ('A Plot Example') >> grid >>も Theprevious set of commands will create plot >>告 labelaxes， write a legend， title and grid the >>告 plot>> exit も finishedwiヒhMATLAB

This completes the first session with MATLAB. Additiona1 commands and features will be encountered throughout the book. In出issession， it is evident白紙 MA礼.ABal¥ows easy manipulation of matrices， definitely in relation to other programming languages. Plotting is not difficult either. These advantages ar芭quitesubstantial in the subject of optimization. 1n出巴 nextsession， we wil1 use the editor 10 I1ccomplish出esame through scripts. This session introduced

• MATLAB Command window and Workspace

・Variableassignment

• Basic matrix operations

• Accessing rows and columns

• Suppressing echoes

• who， inverse commands

• . * multiplication

• Basic plotting commands

1.2.4 Using the Editor

In this section， we will use the editor toα回 te叩 drun a MATLAB a script file. Norma1ly，出eeditor is used to generate two kinds ofMA百.ABfiles.百1凶 efiles are termed scriptftles andfunctionftles. AI出oughbo出ofth巴sefiIes con凶nMATLAB commands like山eoneswe have used， the second type of fiIes needs to be organized in a specified format. Bo曲file ty戸sshould have the extension .m. Although these files are ASCII text fiI凶，山egeneric .m should be凶 edbeca凶 eMATLABs国民hesfor也isex蜘 sion.τhisextension is 山首quetoMA百.AB.百1cscript file contains a Iist of MA TLAB commands白紙areexe印隠din sequence. This is different告。mthe interactive session of the previous section where

34 INTRODUCTION

MAτ1.AB悶卯ndedto回 chconunand immediately. The script file is more useful when

there ar巴manyconun叩 ds白紙 needto be ex釘 utedto ac∞mplish some objective， like running an optimization technique. It is important to remember that MATLAB aIlows you to switch back ωinteractive mode at any time by just typing commands in白eworkspace

window like in曲ep問vioussection.

MATLAB provides an editor for creating these files on the PC platform. This editor

uses color to identify MATLAB statements and elements. It provides白ecurrent vaIues ofthe variables (after白eyare available in the workspace) when the mouse is over

the variable name in the editor. Thel官邸官 twoways to access山eeditor through the

MATLAB Command window on the PC. 5tart MATLAB. This will open a MATLAB

Command or Workspace window. In this window the editor can be started by using

the menu or血etoolbar. On the File menu， click on New and choose M-fiIc. Altemately， click on the Ieftmost icon on the toolbar (出etooltip reads New File). The

icon for the editor can aIso be placed on the desktop， in which case the editor can be

started by double-clicking the icon. In this event， a MATlAB Command window will

not be opened. The editor provides its own window for entering出巴scriptstatements.

At this point we are ready to use the editor. Make sure you即 ad出巴commentsand

understand them， as these procedu問 swill be used often. Using the editor implies we

will be working with m-files. The commands ar巴出esame as in 1h巴interactivesession except there is no MATLAB

prompt prefixing the expressions. To execute these commands you wiIl have to save them toafil巴.Let us call the filescriptl.m. The.m extension need not be typed ifyou are using由巳 MATLABeditor. You can save the file using the Save or Save As

command from most editors. It is import加 tto know the full pa曲 to白isfile. Let us

assume the path for出efile is C:¥OpCbook¥Chl¥scriptl.m. Note出創出epa白 hereis

specified as a PC path description. The reason we need山isinformation is to inform

MATLAB where to find白efile. We do出isin白eMATLAB Command window.

This implies we should have白eMATLAB Command window open. On PCs we start

MATLAB through the icons. In this book we will accomplish most tasks by typing in

the Command window or through programming. ln many instances山ereare altemate

ways using menu items or白etoolbar. This is left to the reader to experiment and

discover. In the MATLAB Command window us巴山eaddpath command to inform

MATLAB of the location of the file:

>> addpath C:¥Opt_book¥Ch1¥

The script th81 will be created and saved in SCT，伊tl.mcan be run by typing (note that

出eextension is omitted)

>> scriptl

To understand and associate with the prograrnming concepts embedded in. the script， particularly for persons with Iimited prograrnming experience， it is recommended to run the script after a block of statements have been written ra山erthan typing山efile

1.2 INTRODUCTION TO MATLAD 35

in its entirety. Prior to running the script you will have to save the script each time 50

白紙 thechanges in由巳fiJeare recorded and the changes are current. Another

recommendation is to deliberately misり'pesome statements and attempt to debug the

error generated by白eMATLAs debugger during回目ution.

Creating the Scr，伊tM-・.flle(The following will be typedlsav吋 ina fil巴.)

も exampleof using script

A1 [1 2 31;

A2 [4 5 61;

も thecommands not terminated with semi-colon will

も displayinformation on the screen

A = [A1; A2]

B [A1' A2']

C A*B

も nowre-create the matrix and perform matrix

者 multiplicationas in other programming languages

者 exampleof for loop

for i 1 3 者 variablei ranges from 1 to 3 in も stepsof 1 (default)

a1 (1， i) i;

end 告 loopsmust be closed with end

a1

for i 6:-1:4 も note loop is decremented

end

a2

a2(1，i-3) i; も fillingvector from rear

者 creatingmatrix A and B (called AA and BB here)

for i 1:3 AA(l， i) a1 (1， i) ;も assigna1 to AA.

AA(2， i) a2 (1， i) ;

BB (i ， 1) a1 (1， i) ;

BB ( i ， 2 ) a2 (1， i) ;

end

も thesame can be accomplished by AA (1，:) a1

者 withoutthe for loop

AA 巷 print the value of AA in the window

BB

who も listall the variables in the workspace

者 considercode for Matrix multiplication

% which Matlab does so easily

36 INTRODUCTION

告 multiplytwo matrices (column of first maヒrixmust 者 matchrow of second matrix) szAA size(AA) も sizeof AA szBB size (BB) ; if (szAA(l，2) szBB(l，l)) 告 onlyin column of AA match the rows of BB

for i ~ l:szAA(l，l) for j 1:szBB(l，2)

CC ( i ， j ) 0 . 0 ;苦 initializevalue to zero for k 1:szAA(l，2)

end

end

CC(i，j) c CC(i，j) + AA(i，k)*BB(k，j); も k -loop

も j -loop end

end者

CC

苦 i-loop if -loop

者 Notethe power of MAT~B derives from its ability to も handlematrices very easily 者 thiscompletes the script session

Save the above fi)e (scriptl.m). Add the directory to the MATLAB path凶 indicatedbefore. Run the script file by typing scriptJ at the command prompt. Thc commands should all execute and you should fmally see the MATLAB prompt in thc Command window.

Note: You can always rcvert to the intcractive mode by directly entering commandsin出eMATLAB window after the prompt. In the Command window:

>> who >> clear C % discards the variable C from the workspace >>者 usewith caution. Values cannot be recovered >> help clear >> exit

This session il1ustrated:

• Usc ofth巴editor

・Creatinga script

• Running a scIipt

• Error debugging (recommcndcd activity)

・Programmingconcepts

. Loop constructs ， if and for loops

・Loopvariable and increments

・Arrayaccess

寄

1.2 INTRODUCTION TO MA礼AD 37

• Clear stalement

1.2.5 Creatlng a Code Snlppet

In this section， we will examine the 0山ertypc of m-fil巴whichis cal1ed thefunction m-file. For those familiar wilh other programming languages Iike C， Java， or F'ORTRAN，出esefiles represent functions or subroutines. They are primarily uscd t'O handle s'Ome specific types of calculations. Thcy also pr'Ovide a way for the modular development of code as wellぉ codereuse. These code modules ar，巴 usedby being callcd or refcrred in other sections of the codc， say through a script file we looked at earlier. The code that calls thc function m-file is called the calling program/code. The essential paramctcrs in dcvel'Oping the function m-filc arc (1) what inpu1 is necessary for the caIculations， (2) what specific calculations must take place， and (3) what information must be retumed to the calling pr'Ogram. MATLAB requires the Slructure of出efunction m・fileto follow a prescribed format.

We will use 1he edi10r 10 devel'OP a func1ion m-file tha1 wiI1 pcrform a polynomial curvefit. II requires a set of砂 data，represen1ing a curve that needs to be fi1， 10ge1her wi出 theorder of the polynomial to be fit. This exercisc is callcd curve fiuing. In Chaptcr 6 such a problem will be iden1ificd as a problem in unconstrained

optimization. For now， the calcula1ions necessary to accomplish the exercise are considered known. It involves solving a linear equation with出enomtal matrlx and a right-h佃 dvcctor obtained using the data points.τ'he output from the m-file will be 山ecoe鉦icientsrepresenting白epolynornial.

Before we start to develop the code， the first line of this file must be formatted as specified by MATLAB. In the first line， the first w'Ord startirig合um出efirst column is 山cwordfunction. It is followed by the set of relum parameters (retumval). Next，山ename (mypolyfit) 'Of the function with the parameters passed to thc function within p抗仙eses(XY，N).明lefile must be saved as name.m (mypolyfit.m). The comme略be1ween the firs1 line and the firs1 executablc statement wiI1 appc釘 ifyou 1ype help lIame (help mypolyfit) in thc Command window. The reason for thc narne mypolyfit.m is that MATLAB has a bui1t司 infunclIon polyfit. Open the editor 10 create the file containing the following information:

function returnval mypolyfit(XY， N) 者

も Thesecomments will appear when the user types 者 helpmypolyfit in the Command window 事 Thisspace is intended to inform the user how to も interactwith the program， what it does 苦 whatare the input and output parameters も Leastsquare error fit of polynomial of order N 者 xy -Data found in XY 告 returnsthe vector of coefficients starting from % the constant term

~ . i: j'J '~ S ~-:t'~!{j!'ょに:it吋よ，~\.J\".'l\"~' ~ I ¥ ，~ f:，~'，，，. j，~ ;'刊;夜長翻最麗~革担躍盟E司..

\U! t:J.\i. ~~t・n::日九り~!，;:)_.~，r..i:1 :1・'.

39

end 者 endi 100p も ifthe x-points are distinct then inverse is not a も prob1em

1.2 INTROOUCTION TO MAl1.AB lNTRODucnON

も

まori 1:N+1 a(i) = 0.0; も initia1izethe coefficient to zero

end

38

returnva1 inv(A)*b';

Save the file as mypolyfu.m. To use the function we will need to create some xy data 阻 dthcn call the mypolyfit function. S凶 MA1LABin白cdirectory伽，tmypolyfit.m resides in. or add the directory to the path. In the MA11.AB Command window: The fol1owing cod巴istyped in the command window.

弓

JA X

*

勺

3x

* x

x

*

nU 2J

+

.，.，

o

X

2

門

4

=

=

10‘J

)

司，白勾ム吋，u

z

，，，，，

・1

・1

・1

・司

a-

，E‘，，、

=噌・晶司4

r

y

y

oxxx

aι

品M

〉

n

〉

e

sz size (XY) ; NDATA sZ(l，l); 者 numberof data points -rows 0王

者 xymatrix if NDATA == 0

fprintf('There is no data to fit'); reヒurnva1 a; 者 zerova1ue returned return; % return back to ca11ing program

end

>>coeff mypolyfit(XY1，3) > > 告 acubic po1ynomia1 was de1iberate1y created to >> % check the results. You shou1d get back > > 者 thecoefficients you used to generate the curve >>者 thisis a good test of the program >>者 Letus create another examp1e

if NDATA < 2*N fprintf'( ， Too few data points for fi t' )

returnva1 a; reヒurn

end

>> XY2 ( : ，1) XYl( : ，1) ;も samefirst co1umn >> XY2 ( : ，2) 2，0 + exp (XY2 ( : ，1) ) > > 者 notethe power of Mat1ab -a vector fi11 >>者 anew set of xy-data

The processing starts here. The coefficients are obtained as solution to the Linear A1gebra prob1em [A] [c] [b] Matrix [A] is the Norma1 Matrix which contains the sum of the powers of the independent variab1e

[A] is symmetric

色‘usmuadu

色

古

島

市

O菅

>>coeff1 mypolyfit(XY2， >> help mypoly王it>>者 youshou1d see the statements you set up > > も Inthe next exercise we wi11 create a script >>も fi1ethat wi11 run a program for po1ynomia1 >>者 curve-fittingwe wi11 save XY2 so that it can >>者 beused again

again 者 cubicpo1y. 3) も 100pover al1 data points 者 variab1eis "11" (e1) (e工)

XY(11，2)*XY(11，1)(i-1);

11

for i 1:N+1; b(i) 0.0; for 11 l:NDATA;

b(i) +

も 100pb(i)

end

save C:¥Opt_book¥Chl¥XY2.dat -ascii ・doub1e者 thiswi11 save the fi1e as an ascii text fi1e も withdoub1e precision va1ues

〉

〉

〉

〉

〉

〉

This conc1udes白eexercise where a code s凶p阿 W凶 writtento回 lc叫ate出E

C∞fficients of the polynooual used to fit a curv巴tosome砂 data.百 etype of filc is thc

for j 1:N+1; if j >= i者 ca1cu1atingupper diagona1 terms

power (i-1) + (j・1); A(i，j) 0.0 者 initializefor k l:NDATA; 告 sumover data points A(i，j) = A(i，j) + XY(k，l)power;

end 苦 kloop end 告 c10seif statement

Aけ， i) = A(i，j) 昔 exp10itingMatrix symmetry

end 苦 endj 100p

40 INTRODUCTION

fimction m-file. It n民地 ωbeused in a certain way.百lecode was臨 tedusing a cubic

polynomial. Nonpolynomial data wcre a1so tested. The data were saved for later use.

1.2.6 Creatlng a program

h出issection a program白紙will陀ad砂 data，curvefit the data using a polynomial， and ∞mpare恥 originaland fi臨 dda泊伊pru回 llywill be developed. The也tawill be read using a User In低出国(UI)window and a dialog box. The code will be developedぉ ascript filc. This gives us an opportunity to revisit most of the code we used回 rlierfor 問inforcement.Wc will also use thc function m-fiIe created in由巳previousexercise.

百ler芭出窓severalways for you or the users to interact with the code you develop.百1巴

basic method is to prompt users for information at恥 promptin the Command window. 司自is出equickest. This is probably what you will use when developing the code. Once

the∞de has been tested， depencling on usefulness it might be relevant ωconsider using

more sophisticated custom eIements Iike input boxes and file selection boxes.百lisbook will continue to use these elements throughout as appropriate. While the input elements us叫 in山iscode arc ncw commands， the rest of the program will mostly use commands 伽 thavc bccn introdu田 d回 rlier.In s叫uentialorder the evcnls in出sprogram are: (1) to read the砂 datasaved earlier using a filc sclection box， (2) to read曲eorder of fit using an Input dialog box， (3) to use the mypolyfit function dcveloped in the last section to obta泊thec田 fficients，(4) to obtain白ec∞必ina凶 ofthe fittcd c町 ve，(5) to graprucally compare血巴original叩 dfitt凶 data，and (6)ω氏陣ton白efitted a∞uracyon曲efi思m

itsel五百巴newscript file will be called prog...J1j弘m.

Start the cditor to create the filc called p'可

告 programfor fitting a po1ynomia1 curve to xy data 者 from告 App1iedOptimization using Mat1ab 者 Dr. P.Venkataraman 若

者 Chapter1， Section 1.2.6 者 Theprogram 100ks for a fi1e with two co1umn ascii 者 datawith extension .dat. The order of the curve is 者 obtainedfrom user. The origina1 and fitted data 苦 arecompared with re1evant information disp1ayed on 者 thesame figure. The program demonstrates the use も ofthe fi1e se1ection box， an input dia10g box，者 creatingspecia1 text strings and disp1aying them

[fi1e，path] =uigetfi1e('*.dat'，'A11 Fi1es'， 200，200); 者 uigetfi1eopens a fi1e se1ection box 者 checkouthe1p uigetfi1e 者 thestring variable fi1e will hold the filename

1.2 INTRODUCTION TO MATLAB 41

苦 thestri叩 variab1epath wi11 have the path も informationthe defau1t directory pointed wi11 者 differdepending on the p1atform

if isstr(file) 者 ifa fi1e is se1ected 10adpathfi1e ['load' ，path fi1e]; 者 10adpathfi1eis a stどingvariab1e concatenated も withthree strings "10ad "， path and fi1e 者 notethe space after 10ad is important

eval(10adpathfi1e);

も eva1uatesthe string enc10sed -which inc1udes the も Mat1abcommand 10ad. This wi11 import the xy-data 告 thedata wi11 be avai1ab1e in the workspace as a 者 variab1ewith the same name as the fi1ename

告 withoutthe extension (this assumes you se1ected も thexy-data using the fi1e se1ection box)

newname strrep(fi1e，' .dat' ，");

も newnameis a string variab1e which contains the 告 stringfile stripped of the .daヒ extension. Thi白

苦 isa string rep1acement command

x eval(newname); 昔 assignsthe imported data to x 者 justfor convenience. The above step is not 者 necessary

NDATA length(x(:，1)); 苦 numberof data points clear path 10adpathfi1e newname

者 getrid of these variab1es to recover memory end

者 Note: if a fi1e is not se1ected， nothing is being % done

者 Useof an input dia10g box to get the order 者 ofpo1ynomia1 to be fitted

PROMPT {'Enter the Order of the Curve'}; 者 PROMPTis a string Array with one e1er時抗告 notethe curly brackets

TITLE 'Order of the Po1ynomia1 to be Fitted'; 者 astring variab1e LINENO 1; 苦 adata variab1e

getva1 inputdlg(PROMPT， T工TLE，LINENO);

42 INTRODUCTION

者 theinput dialog captures the user input in getval

告 ge七valis a string Array 者 checkhelp inputdlg for more information

no str2num(getval{l，l}); 者 thestring is converted to a number- the order

clear PROMPT T工TLEL工NENO 者 deletingvariables

% call function mypolyfit and obtain the coefficients

coeff mypolyfit(x，no);

者 generatethe fitted curve and obtain the squared

告 errorerr2 0.0 i' for i 1 : NDATA 者 foreach data point

for j l:no + 1 a(l，j) = x(i，l)"(j-l);

end y(i) a*coeff; 者ヒhedata forヒhefitヒedcurve err2 err2 + (x(i，2) -y(i))*(x(i，2)-y(i)); 者ヒhesquare error

end

者 plottingplot (x ( : ， 1) ， x ( : ，2) ，， ro' ， x ( : ，1) ， y，， b-' ) ;

% original data are red o's 告 fitteddata is blue solid line xlabel ( ， x' ) i

ylabel ( ， y' ) ; strorder setstr(num2str(no)); 者 convertthe order of curve七oa string 巷 sameas getval if you have not cleared it 者 setstrassigns the string to strorder titlestr ['polynomial curvefit or order'， s七rorder， 'of file'， file)i 者 thethree dots at the end are continuation marks 者 thetitle will have the order and the file name

title (titlestr) legend('original data'， 'fitted data');

errstrl num2str(err2); errstr2 ['squared erどor errstrl]

gtext(errstr2) ;

1.2 INTRODUCTION TO MATlA9 43

告 thisplaces the string errstr2 which is obtained 者 bycombining the string 'squared error' with も thestring representing the value of the error，

者 whereverthe mouse is clicked on the plot. 者 movingthe mouse over the figure you should 告 seelocation cross-hairs

clear strorder titlestr errstrl errstr2 a y x i j clear NDATA no coeff XY2 file err2 getval grid

者 Thisfinishes the exercise

Run the progra四 byfirst running MATLAB in由巳directorywhere these削esare. or adding the p柚 to1∞ate白白血 At白ecommand prompt type proLP耐.The program should EXEcute requiring user input through the tile selection bOLinput dialog， and final1y displaying Figure 1.11.百leappe釘ancemay be slightly different depending on恥 plaぜ'ormMATLAB is being run.

M This finishes the MATLAB section of the chapter.Tlm section lIas introduced ATLAB in a robust manner. A broad mng巴ofprogmmming experience has been

mltlated in this chapter-AH mw commands have hen ideruincd with a bdef explanation in白巴 comments.It is important白紙 youuse the OPPOltunity to type in 出ECodE yourself;That is the only way the use ofMAT1AB will become familiar.The practice a1so willlead to fewer syntax erro瓜 Thewriting of code will significantly improve your abi1ity to debug and troubleshoot. While白ischapter employed a

Polynomial curve fIt of order 3 of fileXY2.dat 5

x

Flgure 1.11 Origlnal and fllled dala.

t¥2J!.:;l;'f.l草山記ω」i44uuム凶1uωi即位kJt:iム幽必泌叫抽融柏崎州附

44 INTRODUCTION

seoarate section on the use of MATLAB out of n町民sity，sub叫 uentchapmMe

CMa伽 izedby a compl仰向rationofthe us関m巴叩ofMAT官.TLAB

optimi匂za低tiぬon.

PROBLEMS

uruuz;ztm叩:;官官a官lrι;叩 ::;t;mathematical modeJ.

1.2 Identify several possible optimization problems related ωan aircraft. For each problem identify allめedisciplines白紙 willhelp establish the mathematical

model.

1.3 Id由巴nt仲 s総ev刊巴叫 possible0叩ptimizap戸m帥州0油blふふ i耐d白e凶匂 a叫11t侃hed血i也s叫ci増plin巴郎s削 w叫illけ凶山川h恥刷1惚叫叫el如pes酬 i凶slぬh恥 ma闘la伽 na削at山i比L刷

modeJ. . 1・4 Identify severa1 possibl巴optimizationproblems related to a microsystern used

for control.ForBach problem identify all thEdiscIPlines that will help establish

出emathematical model. 1.5 Define a problem wit1I respect to your investment in the stock market. Describe

血enatureofthemathematical modd-

1.6 Define由巳 problemand establish由巳 mathematica1model for the I-beam

holding up an independent single-family home.

1.7 Define the problem and identify the mathematical model for an optimum

overhanging traffic light.

1.8 Define血eproblem and identify a mathematica1 model for scheduling and

ootimization of the daily routine activity.

ωD叫ne出ep州 emfor a lamin

命i灼v叩 given aspe配:ei出白cp卯ump.

1・10Define a chemical engineering problem to mix v釘iousmixtures of 1imited availability to make specified compounds to m田 tspecified d巴mands.

2

GRAPHICAL OPTIMIZATION

川叩日山島町やPU12川世話白川

This book includes a large number of cxamples with two variables. Two-variable

problems can be displayed graphically and th巴solutionobtained by inspection. Along with the usefulness of obtaining the solution without applying

math巴maticalconditions， the graphical representation of the problem provides an

opportunity to explore the geometry of many of the numerical techniques we

examine later. This is necessary for intuitively understanding the algorithm and its

progress toward the solution. In practical situations which usually involve over

two variables， there is no opportunity for neat graphical correlation because of the Iimitations of graphical representation. It wil1 be necessary to review the numbers to decide on convergence or the lack thereof. What happens with many variables

is an extension of the geometric features that are observed in problems involving

two variables. A good imagination is an ess巴ntialtool for these problems with

many variables. Contour plots provide the best graphical陀 presentationof the optimization

problem in two variables. The points on any contour (or curve) have the same value

of the function. Several so抗warepackages are available to create and display these

plots. MATLAB. Mathematica， Maple， and Mathcad are among出egeneral-puqJOse

software packages白紙candraw contour plots. In this book， we will use MATLAB for graphica1 optimization.

引が

19日U

1m

が

自

2.1 PROBLEM DEFINITION

The standard format for optimization problems was established in Chapter 1. lt is

rein町oduc叫 herefor convenience:

46 GRAPHICAL OPTIM'ZATION

Minimize fl..xJ， X2， . • • ， Xn)

Subject to: h，(x" X2， • • • ， Xn) = 0

h2(x，・X2・・..，x，，)=O

ht(XI，X2・・...X，，) =0

gl(X"X2・・...Xn)三o

g2(XI.X2・・..，Xn) SO

glll(X"X2・・...x，，)SO

x:gxi三X7. i=I，2，...，n

(2.1)

(2.2)

(2.3)

(2.4)

In山ischapter， while adhering to the format. the necessity for zero on the right-hand

side is relaxed.百 isis being done for convenience and 10 aid comprehension.百le

right-hand side can also have numerical values other 1han zero. Th巴 firstexample

chosen for illustration is a simple onEusing elementary functions whose gmp12ical

nature is well known. This simple example will permit exarninntion of the MATLAB

code白atwill gencrate曲ecurves and the solution. It will also allow us to define山E

format for the disp1ay of solution to the graphical optimization problcm.

2.1.1 Example 2.1

The first example， Example 2.1， will have two equality constraints and two inequality

constramts:

Minimize fl..XI' X2) = (XI -3)2 + (X2 -2)2 (2.5)

Subjcct to: hl(xl. X2): 2x1 + x2 = 8 (2.6a)

h2(x" XU: (XI -1)2 + (ゐ-4)2=4 (2.6b)

gl(X" X2): Xl + x2 S 7 (2.7n)

g2(x，. XU: XI ー 0.25~SO (2.7b)

o SXI S 10; OSX2 S 10 (2.8)

In the above dcfmi1ion， we have two straight lines. two circles， nnd a parabola. Note

that two equality cons回 intsand two variables imply伽 t出E陀 isno scope for

o出凶zation.百leproblem will be detcrmined by出巴司uali1yconstraints provided the

twocons回 intsare Iinearly independent， which is true in this exnr叩le.This example

wasα凶 1ed10 help understand由巳codcinMA'礼 AB山atwill be used to draw graphical

solutions in this book.

震芸

2.1 PAOBLEM DEFINfTlON 47

Figure 2.1 iIIustrates the graphical solution to白isproblem.百lefigure also displays additional information， related to the inequality cons佐aints，placed on it after the principal graphic information was generated and displayed.

2.1.2 Format for the Graphlcal Dlsplay

The graphical solution to Example 2.1， as seen in Figure 2.1， is generated using MATI.AB [1.2] ex∞:pt for the identifi伺 tionof the inequality constraints. All inequality constraints are distinguished by hash marks. The hashed side indicates the infeasible region. In MA'τ1.AB 5.2，出ereis no feature to insert these marks through programming. An add-on program called Matdraw was used to create the hash Iines in the figure. It can be downloaded from the Mathworks site. Beginning wi山Version5.3 there is a figure (plotedit)巴ditor白紙 willallow you to insert additional graphic elements to出Efig町e.百lestudent can also p叩 cilin由民ehash tines after obtaining the printout It is to be noted出atthe graphical solution is incomplete if the inequality constraints are not distinguished， or if the feasible region has not been established in some manner.

10

g

B

3

。。 2 3 4 56 x1 valu国

7

円guro2.1 Graphical solution for Example 2.1.

s 10

48 GRAPHICAL OPTIMIZATION

In the graphical display of solutions， only血eobjective function is drawn for

several contours so that白edirection for白eminimum can be identified. Each cquality

constr必ntis drawn as a single curve for出enumerical value on the right-hand side.

They should be identified. Each inequality constraInt is similarly drawn for a value on

出eright-hand side. They need to be identificd too. The hash marks ar町 lrawn/penciledin on the figurc or a printout of出efigu陀.The extent of the dcsign region is established

and the solution identified on the figure. The relevant region can be zoomed for better accuracy.百四 data for白eplot are obtained using MATLAB'S matrix operations after

establishing the plolting mesh. This exploits MATI.AB'S nalural speed for matrix

operations.

2.2 GRAPHICAL SOLUTION

MATLAB poss凶 sa powerful visualization engine that permits the solution of白e

two-variable optimization problem by inspection. There are three ways to lake

advantage of the graphic features of MATI.AB.百】efirst is lhe use of MATI.AB'S

high-Ievel graphing routines for data visualization. This will be山巴 primaryway to

solve graphical oplimization problems in this book. This wil1 also be出eway 10 incorporate graphical exploralion of numerical techniques in出ebook. For mo隠

precise control over the display of data， MATLAB allows us巴rintcraction through

programming using an object-oriented system identified in MATI.AB as Handle

Graphics. The third use ofthe MATLAB graphics engine is to use the Handle Graphics system to develop a Graphical User Interface (GUI) for the program or m-file. This

creales a facility for sophislicated user interaction. MOSl of the plotting needs can be

mel by the high-Ievel graphics functions available in MATLAB.

2.2.1 MATLAB High-Level Graphlcs Functlons

τ'here are three useful windows during a typical MATLAB scssion. The first is白eMATLAB Command window through which MATLAB receives instruction and displays alphanumeric information.百lesecond window is the texl-editor window where

m-fiIes are coded. The third is the Figure window where the graphic elements are displayed. There can be more白anone figure window. The figure window is出巴targel

of the high-level graphics functions. The graphics functions in MATLAB allow you to plot in 2D or 3D. They allow

contour plots in 2D and 3D， mesh and suばaωplots，bar， area， pie charts， histograms， animation， and gradient plots. Subplots can also be displayed using these functions. In addition they permit operation with images and 3D modeling. They allow basic control of thc appear阻 ceof the plot through color， line style， and markers，蹴is

ranges， and鎚 pectratio of the graph.τ'hey permit annotation of the graph in several

ways. Some of these functions will be used in the next section when we devclop the

2.2 GRAPHICAL SOLUTION 49

:.L

m-file for山efirst example. The following introduces some more useful Inform姐onabout the use ofMATLAB・8high-Ievcl graphics functions.

The two main graphical elements白紙arctypically controlled using the high-Ievel

graphics functions are the figu陀 andthe axes. Using Handle Graphics you can control most of the other graphical elements， which include elem巴ntsused in the GUI. These other elements are typically childrcn of山eaxes or figure. The figure function or command creates a figure window with a number starting at one， or will creat巴anewfigure window incrementing the window count by one. Normally all graphics functions are targeted to the c町rentfigur芭 window，which is selected by clicking it with the mouse or executing the command figure (number)， where number is the number of the figure window山alwill have lhe focus. AIl commands are issued in command window. Graphics functions or commands will automatically create a window if none exists.

Invoke help commandname at the MATLAB prompt to know more about the functions or comrnands. The word handle app回路 onm加 yplaぽormsand in many

applications， particularly those that deal with graphics. They are widely prevalent in object・orientedprogramming practice. MATI.AB・svisualization system is object oriented. Most graphical elements are considered as objects. The “handle" in MATI.AB is a sy印刷oftwarecreated number白紙 canident汀'ythe specific graphic object. If this handle is available，山enproperties of the objecl such as line size， marker type， color， and so on can be viewcd， set， or reset if neccssary. In MATLAB， Handle Graphics is出eway to custornize出egraphical elements. In high-Ievel gr叩hicsfunctions出 sis used in a limited way. In this chapter， we will use it in a minimal way to change出E

charactcristics of some of the graphical elements on the figure.

To understand the concept of "handlcs" we will run出efollowing code interactively. It deals with creating a plot， which was introduced in Chapter 1. The boldface comments in the code segment indicate the new fealures that are being emphasized.

Sta武MATLABand interactiveLy perform the following:

>>X O:pi/40:2*pi; >>y X.合sin(x) ; >> plot(x，y，'b-'); >> grid;

>>h plot(x，y，'b-') 者 h is the handle to the plot >>告 anew plot is overwritten in the same figure window > > 苦 anumerical value is assigned to h >>者 Wecan use the variable name h or its value >>告 torefer to the plot again

>> >> set(h，'LineWidth' ，2); も thisshould make your plot

>> 告 thickerblue >> set(h，'LineWidth'，3，'LineStyle'，' :'， 'Color' ，'r')

r

o

可ム

。c e

r

h

0

・D

t

r

con

et--

vc

ex

xv s

eyv

ト

Laey

eヒ

rat

ceo

r

l

も

c

p

也

電

也

電

50 GAAPHICAL OPTIMIZATION

>>者 Thehandle is used to refer to the object >>も whoseproperty is being changed >>者 UsuallyProperty information occurs in pairs of >>者 property-namejproperty-value>> も property-valuecan be a text string or number >>

>> get(gca) も thiswill list the property of the axes > >者 ofthe curren七 plot.Note there are a significant >> 者 amountof properties you can change to customize > >者 theappearance of the plot

>>

>> set(gca，'ytick'， [-5，ー2.5，0，2.5，5]) > >告 youhave reset the ytick marks on the graph >> set(gca，'FontName'，'Arial' ，'FontWeight'， ... >> 'bold'，'FontSize' ，14) >>

> >者 Changesthe font used for marking the axes >> set(gca，'Xcolor' ，'blue') > >も changesthe x-axis to blue >> % concludes the demonstration of handles

From the session above it is evident白紙 tofine-tune the plot you create， you need to first identify the object handle. You use the handle to access白eobject prop凶 y.You change a property by setting its value. We will see more graphics

function as we obtain the graphical solution to Example 2.1 in由巳 following

sectlon.

2.2.2 Example 2.1ー-GraphicalSolution

Figure 2.1 is the graphical representation of Example 2.1. The range for the plots matches the side constraints for the problem. The intersection of the two equality constraints identifies two possibl巴solutionsto the problem， approximately (2.6， 2.8) and (1， 6). The inequality constraint g2 makcs the point (2.6， 2.8) unacceptable. Point (1， 6) is acc巴ptablewith respect to both constraints. Thc

solution is therefore at (1， 6). While the objective function was not used to determine thc solution， contours of the objective function are drawn indicating白巴

direction for出eminimum of the objecti v巴function.The solution can be identified by inspection of the ass巴mbledplots without reference to the terminology or the

techniqucs of optimization. Recall， however. that this is only possible with one or two design variables in the problem.

The code for血isex創npleinvolves six m-files: a script m-file白紙 willinc¥ude the MATLAB statements to create and display出edata， and five m-fi1国一onefor

cach function involved in Example 2.1. We begin with the script m-file-

Ex2_1.m.

2.2 GAAPHICAL SOLUTION 51

File: Ex2 1.m

者 Chapter2: Optimization with MATLAB % Dr. P. Venkataraman 者 Example2_1 (Sec 2.1-2.2) 者

もgraphicalsolution using MATLAB (two design variables) 毛 thefollowing script should allow the display 苦 ofgraphical solution 苦

も Minimizef(x1，x2) 者 h1(x1，x2)者 h2(x1，x2)告 gl(X1，x2)告 gl(x1，x2)苦

(x1・3)**2 + (x2・2)**22x1 + x2 8

(x1-1)2 + (X2-4)2 4

x1 + x2 <= 7

x1 -0.25x22 <= 0.0

% 0 <= x1 <= 10 0 <= x2 <= 10 者

も NOTE: The hash marks for the inequality も constraintsmust be determined and drawn outside 者 ofthis exercise and on the printout 者ーーーーー---・・白ーーーーーーーーーーー-- ーーーーーーーーーーーーーー"・ー・"ーーーーーー--

x1=0:0.1:10i 者 thesemi ・colonat the end prevents % the echo

x2=0:0.1:10i 者 theseare also the side constraints 者 x1and x2 are vectors filled with numbers starting 苦 at0 and ending at 10.0 with values at intervals 告 of0.1

[X1 X2] meshgrid(x1，x2)i 者 generatesmatrices X1 and X2 corresponding to % vectors x1 and x2. It is a mesh of x1 and x2 values 者 atwhich the functions will be evaluated f1 obj_exl(X1，X2) i告 theobjective function is 者 evaluatedover the entire mesh and stored in f1 者 MATLABwill compute the values for the objective も throughafunction m田 filecalled obj_exl.m ineq1 ineconl(Xl，X2)i 巷 theinequality gl is

者 evaluatedover the mesh ineq2 inecon2(Xl，X2)i 者 theinequality g2 is

も evaluatedover the mesh eq1 = eqcon1(Xl，X2) i 者 theequality h1 is evaluated

者 overthe mesh eq2 = eqcon2(X1，X2) i 告 theequality h2 is evaluated

者 overthe mesh


[C1，han1] contour(χ1， x2， ineq1， [7，7) ， 'r-' ) i

も asingle contour plot of gl (or ineql) is drawn for 者 thevalue of 7 in red color as a continuous line も duplicationof the contour value is necessary も fordrawing a single contour 者 han1is the handle to this plot. This handle 者 canbe used to change the plot display characteristics 告 Clcontains the value of the contour

clabel(C1，han1) i

も labelsthe contour with the values in vector C1 者 contour(x1，x2，ineq1，[7，7] ，'r-') will draw the contour 告 withoutlabeling the value hold on 告 allowsmultiple plots in the same figure 告 windowgtext ( ， gl ' ) i

も willplace the string 'gl' on the plot at the spot 苦 selectedby a mouse click. This is a text label 者 thisprocedure is repeated for remaining も constraints

[C2，han2] contour(x1，x2，ineq2， [0，0] ，'r-ー， ) i

clabel(C2，han2)i gtext('g2') i

[C3， han3] contour (x1， x2， eq1， [8，8] ， 'b-' ) i

clabel(C3，han3)i gtext ( ， h1' ) i

[C4，han4] contour(x1，x2，eq2， [4，4]， 'b--') i

clabel(C4，han4) i

gtext ( ， h2' ) i

[C，han] contour(x1，x2，f1，'g')i 告 contourof 'f' is drawn in green color. The number 苦 ofcontours are decided by the default value clabel (C， han) i

xlabel('x1 values' ，'FontName' ，'times' ，'FontSize' ，12，. 'FontWeighヒ，，'bold') i も label for x-axes ylabel('x2 values' ，'FontName' ，'times' ，'FontSize' ，12， 'FontWeight' ，'bold')i grid

hold off

The function m-fiIes are:


凪鑑on1.m

function retval inecon1(X1， X2) retval X1 + X2i も X1，X2 are matrices

者 retvalis the value being returned after the 告 computation

者 SinceX1 and X2 are matrices retva1 is a1so a 告 matrix

者 Bythis way the entire information on the mesh 者 isgenerated by a sing1e call to the funcヒion者 inecon1

凪血血2.m

function retva1 inecon2(X1，X2) retval Xl -0.2S*X2. 2i

者 Notethe use of ^ operator for element by e1ement 告 operation. That is each e1ement of the X2 matrix 者 issquared. Without the dot the imp1ication is a 者 matrixmultiplication -between matrices whose 者 innerdimensions must agree. Similar operators are 告 definedfor element by element mulヒiplication者 anddivision

盤姐Lm

function retval eqcon1(X1，X2) retval 2.0*Xl + X2i

魁迎且2.m

function retval eqcon2(X1，X2) retval (X1 -1) .*(X1 ・ 1) + (X2 -4). * (X2 -4) i

曲U叫掴

function retval obj_ex1(X1，X2) retval = (X1 -3) .*(X1 -3) +(X2 -2).合 (X2 ・ 2)i

2.2.3 Displaylng the Graphics

Al1 of lhc files required for graphical display of the problem have been crcalcd. In the Command window， lype addpatb followed by出ecomplele path for thc directory山at

holds these files. At the prompt， type thc name of thc script file for the example without the .m eXlension

>> Ex2 1

54 GRAPHICAL OPTIMIZAnON 2.2 GRAPHICAL SOLUTION 55

The first contour plot will appear. Move the mouse over to白eplot and a cross hair

appears. Clicking on the plot wi1l plac巴thestring "gl" at the cross hair. The second

contour plot should appear and there is a pause to plac巴出etext label. After the four

cons住aints，出eobjective function is plotted for several contour values. Finally the plot should app伺 rωinFigure 2.1 without the hash marks (and wi山yourchoice of白B

location for constraint labels). You can insert the hash marks through editing the plot

directly (Version 5.3 onw釘ds).明記 solutionfor the problem is at (1，6)， where four plots intersect. The value of the objective function is 20鉱山esolution.

If the plot is acceptable， you can print the informalion on the figure by expo凶ngit (using an appropriate extension) 10 a file and later incorporating it in another document. You can also send it to出eprinter from Ihe print command on the File

menu. Typing help print in the Command window should Iist a set of commands you

can use to save曲efile.

10

広説

明剖=官、，

J

。。 2 4 6 X1 values

s 10

>> print -depsc2 plot_ex_2_1.eps

wi1l c路 atethe level 2 color postscript file called plocex_2_l.eps in the working directory. A complete path name should save the file to白especified directory. You

can then execute standard operating system commands 10 dump the filc 10出eapplicabl巴printcr.You can also save the figure (ex蜘 sion.fig) and later open it in

MATLAB using the filc open command from menu b釘 ortoolbar.

2.2.4 Customlzlng the Flgure Flgure 2.2 Customized両日U悶 forE姐 mple2.1.

In this section thc basic figure and plots creatcd abov巴wilIbe custornizcd using the

Handle Graphics commands first explo陀 dinS巴ction2.2.1. All ofthe plots describing 白econstraints will have a linewidth of 2 points.百leobjective function will have a

linewidth of 1 point. The text labels wi1l be in the "Times" font. It wilI be in boldface with a font of size 14 and in白ecolor of the cons回 int.Thex叩 dY tick marks will be changed to establish a broader grid. The x叩 dy labe1s will include a subscript. A two-row text wi1l be inserted at the point selected through the mousc. The ncw figure is shown in Figurc 2.2.

First ∞py the file in出eprevious exercise and rename it.羽lerenamed file will be used ωmakethen配 essarych佃 gesto出ecode. These changes inc1ude both new code as weIl as replacement of existing lines of code. In出efoIlowing only白echanges to

出巴 codeare i1Iustrated. The original code is shown in the italic style and identified

with three ellipsis points (. . .) at出巴 beginningof the line. The code to be inserted or

replacement code is shown in norm叫styl巴.If no replacemcnt is apparent， then the old code is used 10 define the locations whe回目ew∞deis appended.

gt儲 t('gl');

k1 gtext ( ， gl' ) ;告 k1is the handle to the text element set(k1，'FontName'，'Times'，'FontWeight'，'bold'，， FontSize' ，14，'Color'，'red')

..clabel(C2，han2}i set(han2，'LineWidth' ，2) k2 = gtext('g2');

set(k2，'FontName' ，'Times' ，'FontWeight'，'bold'，， FontSize' ，14，'Color'，'red')

clabel (C3，han3); set(han3，'LineWidth' ，2) k3 ~ gtext('h1');

set(k3，'FontName'， 'Times' ，'FontWeight' ，'bold'， 'FontSize'，14，'Color'，'blue')

{Cl，hanl} contour(xl，x2，ineql， {7，7}，'r-'};

set(han1，'LineWidth' ，2); 者 sets1ine width of the gl も contourto 2 points

clabel(C4，han4); set(han4，， LineWidth' ，2) k4 ~ gtext('h2');

56 GRAPHICAL OPTIMIZATION 2.3 ADDlTIONAL EXAMPLES 57

set(k4，'FontName' ，'Times' ，'FontWeighヒ 'bo1d'，， FontSize' ，14，'Co1or' ，'b1ue')

c:label (C， han) ; set(han，'LineWidth' ，1) .. xlabel ( ， xl val ues' ，， FontName' ， 'times' ，

x1abe1(' x 1 va1ues' ，'FontName' ，'times'，， FontSize' ，12，. ， FontWeight' ，'bo1d') i

y1abe1(' x 2 values' ，'FontName' ，'times'，' FontSize' ，12， 'FontWeight'，'bo1d')i set (gca，， xtick' ， [0 2 4 6 8 10]) 者 setxticks seヒ(gca，'ytick'，[0 2.5 5.0 7.5 10]) 昔 setyticks k5 gtext({'Chapter 2: Examp1e 1'，

'pretty graphical display'}) 者 theabove gtext describes a string array も stringarray is defined using curly braces set(k5，'FontName' ，'Times' ，'FontSize' ，12，， Fontweight' ，'bo1d')

!(xI'X2) =町î+bx~ 一 cco中川)ー dcos(qx2) -1-c + d

with

a = 1， b = 2， c = 0.3， d = 0.4， p = 3π，q=4π

(2.9)

Figurcs 2.3-2.7 ar宮山egraphical display of solutions泊出issection. The figures in the book being res凶ctedto black and white will not∞nvey也eimpact of color images you will see on the screen. Thcre are two files associatcd with the plots: the script m-fiIe (ex2.-2.m) and thc function m-fiIe (obLex2.m).

盟主..z.m

者 Chapter2: Optimization with MATLAB 者 Dr. P.Venkataraman 者 Examp1e2.2 Sec.2.3.1 者

(at the end) c1ear C C1 C2 C3 C4 h h1 h2 h3 h4 k1 k2 k3 k4 k5 壱 getrid of variab1es from the workspace

Run the prograrn to sec曲efigure in Fig山宮 2.2.Once the plot is customIzed to your satisfaction， you can make it白巴standardfor other plots you will pr吋 U四 inMA礼As.This file could also be a template for general contour plotting. Note白ata1l function information is obtained through function m-fiIes which are coded outside the script filc. Sctting up ncw problems or examplcs only requires addinglchanging new function m-fi1ω.

間11ωIAbclodConl凹 g

defaull cGlcr

2.3 ADDITIONAL EXAMPLES

The f01l0wing additional ex創npleswill serve to iIlus回 tebo白optimizationproblems鎚

well as additional gmphical fcatures of MA TLA白血atwill be useful in d巴velopinggraphical solutions to optimization problems.明lCgrョphicalroutines in MATLAB are powerful叩 d回 SYto use. They can grョ，phicallydisplay the problems副総vem1ways wi白verysimplc ∞mm阻 ds.The useful display is， however， detcnnined by血euser. The first example in this section， Example 2.2， is a problem in uncons回 inedopt泊前回tion.明記 secondcxample is a structum1 engineering problcm of reasonablc complcxity.百lC白凶exar即 Icdemons回飽soptimization in thc area of heat Iransfcr dcsign.

明岨固岡田、，

2.3.1 Example 2.2

This cxamplc iIIus位置tesscvcral diffcrent ways of gmphicaUy displaying a function of two variables. The problem was uscd to iIIuslrate global optimization in Reference 3. The singlc objcctive function is

-0.5 O XJ valu鎚

0.5

円gu開 2.3 Filled conlours with colorbar. Example 2.2.

59 2.3 AOOmONAL EXAMPLES

2DCon随町

with Grndi.nl V，酎:10悶

GRAPHICAL OPTIMIZATlON 58

82-gJ

X1 values -1 X:!V剖ues

0.5 o X1 values

-0.5

3D conlour wilh ∞lorbar. Ex創叩le2.2.

者 theobjective function is 告 evaluatedover the entire mesh

者 filledcontour with default colormap 者 helpgraph3d gives you the choices for colormap [C1，han11 contourf(x1，x2，f1， [0 0.1 0.6 0.8 1.0 1.2 1.5 1.8 2.0 2.4 2.6 2.8 3.01);

Flgure2.5

f1 obj_ex2(X1，X2); 1臨書y't.

ticks ticks

者 specificcontour levels indicated above clabel(Cl，hanl); colorbar告 il1ustratesthe default color scale set(gca，'xtick'， [-1 -0.5 0.0 0.5 1.01) も customset (gca， 'ytick' ， [・1 -0.5 0.0 0.5 1.01) 者 customgrid

xlabel(' X_1 values' ，'FontName' ，'times'， 'FontSize'， 12);

% label for x-axes ylabel(' x 2 values' ，'FontName'， 'times'，

告 graphicalsolution using MATLAB (two design variables) も Unconstrainedfunction illustrating global minimum も Examplewill introduce 3D plots， 3D contours， filled も 2Dcontours with gradient information 者・ーーーー・・・・・・ーーー・ー・ーーーーーーーーーー------・ーーーーーーーーー------ーー--

x1=・1:0.01:1; も thesemi-colon at the end prevents

" the echo x2=-1:0.01:1i 巷 theseare also the side constraints 者 x1and x2 are vectors fil1ed with numbers starting 苦 at-1 and ending at 1.0 with values at intervals of 者 0.01

Contour wlth gradlent vectors: Example 2.2. 円gure2.4

[Xl x21 = meshgrid(x1，x2) ; 者 generatesmatrices X1 and X2 corresponding 者 vectorsx1 and x2 者 reminderMAT工~ is case sensitive


CO!U'SC Mesh P101 ωlormap・目。1

4

3.5

3

2.6

2

1.5

au

白

u

・，

mw

0.6

X2 values -1 -1

X1 valu凶

Flgure 2.6 3D mesh plot wlth colorbar: Exampl自2.2.

， FontSize' ，12); title({'Filled Labelled Contour'，' ... default color map'}，'FontName' ，，ヒimes'，'FontSize' ，10)

者 anew figure is used to draw the も basiccontour plot superimposed with gradient

告 information告 alsoinformation is generated on a coarser mesh to 者 keepthe figure tidy. grid is removed for clarity

figure告 anew figure window is drawn

y1 -1:0.1:1.0;

y2 ー1:0.1: 1; [Y1，Y2] meshgrid(y1，y2); f2 obj_ex2(Yl，Y2); [C2，han2] contour(y1，y2，f2，

[0 0.5 0.75 1.0 1.5 1.75 2.0 2.5 3.0]);

2.3 ADOmONAL EXAMPlES 61

Coa問。SurfacePlot ∞lonnnp -jcVdefnult

4

3.5

3

2.5

2

1.5

0.5

0 1

0.5

。 0.5 。心5

.0.5

X2 values -1 -1

民HM

-a v

x

Flgure 2.7 3D su巾 ωplotwith default colorbar: Example 2.2.

2

1.5

clabel(C2，han2) [GX， GY] gradient(f2，0.2); % generation of gradient information see help gradient hold on 者 mulヒipleploヒson the same figure quiver(Y1，Y2，GX，GY); 者 drawsthe gradient vectors at grid points も seehelp quiver hold off set(gca，'xtick'， [-1 -0.5 0.0 0.5 1.0]) set(gca，'ytick'， [-1 -0.5 0.0 0.5 1.0]) xlabel(' x_1 values' ，'FontName' ，'times'，

， FontSize' ， 12) ;

ylabel(' x 2 values' ，'FontName'，'ヒimes'， 'FontSize'， 12);

title({'2D Contour'，'with Gradient Vectors'}，， FontName' ，'times'，'FontSize'，10)

62 GRAPHICAL OPTIMIZATlON

も A final example of contour information is a 3D % contour plot -or a stacked contour plot figure colormap(spring) contour3(x1，x2， f1，

[0 0.3 0.6 0.8 1.0 1.5 1.8 2.0 2.4 2.6 2.8 3.0]); set(gca，'xtick'， [-1 ・0.50.0 0.5 1.0]) set(gca， 'ytick'， [-1 -0.5 0.0 0.5 l.0]) も changecolormap and set colorbar 者 changingcolor for display is easy by selecting 者 amongsome colormaps provided by MATLAB colorbar xlabel(' x 1 values' ，'FontName'，'times'，

， FontSize' ， 12); ylabel(' x 2 values' ，'FontName'，'times'，...

， FontSize'， 12); tiヒle({'Stacked(3D) Contour' ，:colormap -spring'}，， FontName' ，'times' ，'FontSize' ，10)

grid

苦ヒhenext two figures wi.ll display 3D plots 者ヒhefirst is a mesh plot of the function 者 oncemore coarse data is used for clarity

figure colormap (cool) 苦 anothercolormap mesh (y1， y2， f2) 者 usinginformation generated earlier set(gca， 'xtick'， [-1 -0.5 0.0 0.5 l.0]) set(gca，'ytick'， [-1 ・0.50.0 0.5 1.0])

colorbar xlabel(' x 1 values' ，'FontName'，'times'，

， FontSize'， 12); ylabel(' x_2 values'，'FontName'，'times'，...

， FontSize'， 12); title({'Coarse Mesh Plot' ，'colormap -cool'}，， FontName' ，'times'，'FontSize' ，10)

grid

者ヒhefinal plot in this series 者 surfaceplot with default colormap figure colormap(jeヒ}surf(y1，y2，f2) 告 usingold information colorbar xlabel(' x 1 values' ，'FontName' ，'times'，.

2.3 AOOrrtONAL EXAMPLES 63

， FontSize' ， 12); ylabel(' x_2 values' ，'FontName' ，'times'，

， FontSize'， 12); set(gca， 'xtick'， [-1 -0.5 0.0 0.5 l.0]) set(gca，'ytick'，1-1 -0，5・0.00.5 l.0]) title({'Coarse Surface Plot'，‘colormap -jet/ default'}，'FontName'，'times' ，'FontSize' ，10)

grid

Q祉s謡.m

function retval obj_ex1(X1，X2) も Optimizationwith MATLAB 者 Dr. P.Venkataraman 者 Chapter2. Example 2.2 者

も f(x1，x2) ョ a*xl""2+ b*x22・c合cos(aa*x1)ー

も d*cos(bb*x2) . . . も+c + d

a '" 1; b 2・ c 0.3; d '" 0.4; aa 3.0*pi; bb 4.0*pi;

者 notematrix operations need a dot operator retval a*X1.*X1 + b*X2.*X2 -c*cos(aa*X1) ー

d*cos(bb*X2) + c + d;

The brief comments in the code should provide an explanation of what you see on 由巳 figure.Figures can be further custornized as secn in血eprevious section. From an optirnization perspective， Figure 2.4 provides thc best infonnation about thc nature of the problem. The 2D contour curves identify the neighborhood of the 10伺 lrninimum.百legradient vectors indicatc山巴 directionof出efunctions steepest rise a1出巴 point，so戸松sand vallcys can bc distinguishcd. The contou悶 thernselvescan be colored without being fillcd.

Thc quiver plp1 shown in Figure 2.4 also provides a mechanism 10 indicate白efeasible region when dealing with inequality constraints sincc they indicatc出edirection in which由巳constraintfunction wiII incr，伺se.If several functions are bcing 合司wn，白enthc cIutter produ田dby白earrows may dぜfuse白ecIarity. The user is encouragcd 10 use the.powerful graphical features ofMATLAB to his benefit at all times without losing sight of出eobjective of his cffort. MATLAB graphics has many more features than will be covered in出ischaptcr. The expos山冠 inthis chapter should bc sufficient for血E隠晶derωconfidentlyexplore many other useful graphiω∞mmands.

Thc 3D mesh and surface plots have limited usefulness. These plots can be uscd to reinforce some of the features found in Figure 2.4. The infonnation in these plots may be improved by choosing a camera angle白紙emphぉlZCS釦measpect of the graphi回 Idescription. This exploration is left to出C陀aderas an exercise. Using help view in the MATLAB Command window should get you startcd in出isdi陀ction.


2.3.2 Example 2.3

百lenext example is a complex one from structural engineering design白紙isrelevant in civillmechanicallaerospace cngin田 ringapplications. It appeared as a problcm in Refercnce 4. It is dcveloped in detail here. The problem is to redesign the basic tall flagpolc in view of the phenomenal increasc in wind spceds during extreme weather conditions. In recent catas佐'Ophicevents. the wind sp田 dsin tomadocs have bccn measured at over 350 miles per hour. These high specds app伺 rto be the norm rather 曲叩佃 unusualcvent.

Deslgn Problem: Minimize the mass of a standard IO-m tubular flagpolc to withstand wind gusts of 350 miles per hour. The flagpole wi¥l be made of structural steel. Use a factor of safety of 2.5 for白cstructuraJ design.百lCdeflection of the top of the flagpole should not exceed 5 cm. The problem is described in Figure 2.8.

Mathematical Model: The mathematical modcl is developed in detail for completencss and to provide a rcview of useful s凶 ctural(5) and aerodynamic relations [6). The relations are exprcssed in original symbols rather than in standard format of optimization problems to provide an insight into problem formulation.

Deslgn Parameters: Thc stnlctucal stcel [5] has the fol1owing material constants:

E (modulus of clasticity): 200 E叫 9Paσ011 (allowablc normaJ s岡崎):250 E+06 Pa 1:011 (allowable shcar stress): 145 E+06 Pa

8m

A

10m

円gure2.8 Flagpole deslgn: Exampla 2.3.

。s8ctlon A，・A

2.3 ADDITIONAL EXAMPLES 65

'Y (materiaJ density): 7860 kglm3

FS (factor of safety): 2.5

g (gravitational acceleration) = 9.81 mls2

For the acrodynamic calculations the following are considered:

p (standard air density): 1.225 kglm3

Cd (合唱 coefficientof cylinder): 1.0 Wp (flag wind load at 8 m): 5000 N Vw (wind speed): 350 mph (156.46 mls)

Thegeome凶cparameters arな

Lp: the location of flag wind load (8 m) L: leng白 ofthepol巴(10m)

OlIll: permiUed deflection (5 cm)

Deslgn Vali俗bles:The design variables shown in Fig 2.8 are

do: outside diameter (XI) [ Note: x's are not used in山emodel] dj: inside diameter (.x2)

Geometric Re伺uons:The fol1owing relations will be useful in later caJculations: A: area of cross却 ction= 0.25吋 *(d~-df)

1: diame甘icalmoment of inertia =π* (d~ -dt)/64

Qlt: first momcnt of area above the neutral蹴 isdivided by thickness = (d~ + dodi + dr)/6

Objective Function: The objective function is the w巴ightof the IO-m uniform flagpole:

Weight:j{xl' X2): L * A * 'Y * g (2.10)

Constra的tFunctions:・百lewind load per unit length (FD) on the flagpole is caIculated as

Fo = 0.5 * P * V~ * Cd * do

Thc bending moment at the base of the pole due to白isuniform wind load on the enti問

pole is

Mw=0.5 * Fo・L*L


The bending moment due to the wind load on the f1ag is

M~= Wn"'L_ F-"""P AJD

Bending (nonnal) stress at the base of白epole is

σ'bcnd = 0.5ホ (Mw+MF)場 dil

Nonnal s位essdue to the weight is

σweight = 'Y事 g地L

Tota1 nonna1 sU芭ssesto be resisted for design is白esum of由巳 nonna1stresses

computed above. Incorporating the factor of safety and the a1lowable stress from materia1 va1ues， the first in叫ua1ity∞nsU湿intcan be set up as

gl(XI'~): σbcnd +σwcight S σnUIFS (2.11)

The maximum shear load in the cross section is

s= WF+Fo '" L

The maximum shear stress in the pole is

τ=S* QI(I取 t)

The second inequa1ity constraint bωed on handling由e油田rstresses in the f1agpole

18

g2(X，，~): τgτ:aIJIFS (2.12)

百lethird practica1 coostraiot is based 00 the def1ectioo of the top of血epole.τ'his

d巴flectioodue to a unifonn wind load on the pole is

ow = Fo 01< L4/(8場 E*乃

The def1ection at the top due to the f1ag wind load atらis

OF= (2ゆWF'"L3_ WF市 L'" L '" Lp)/(E '"乃

The third coostraiot translates to

C3(X1・X2) ow + OF S O.u (2.13)

品

、f/

2.3 AODITIONAL EXAMPLES 67

To discourage solutions where do < dj， we will include a geometric constraint:

84(X1， X2) d，。ー叫ミ0.001 (2.14)

Side Constraints:・百lisdefines the design region for the search.

2 cm S do S 100 cm; 2 cm S di S 100 cm (2.15)

MATLAB Code: The m-files for this example are given below. An important observation in出isproblem， and structural engineering problems in particular， is白巴

order of magnitude of the quantities in the constraining珂uations.百lestress coos釘泊ntsare of出eorder of 108+06， while the displacemeot lenns are of山eorder

of 108-02. Most numerica1 techniques s凶 ggleto handle this range. Typica1ly， such problems 0田 dto be nonna1ized before beiog soIved' It is essentia1 in applying numerical techniques used in optimization.

This example is plotted in two p紅15.In the first part， each inequa1ity COI首位置intis investigated a1one. Two curves are showlI for each cons甘凶nt.This avoids clutter.τ'he

side for the location of the hash mark 00 the constraint is detennined by drawing these C町Vωincolor.百leblue color indicates the feasible direction. Altemately， quiver plots can be used. The second part is山econsolidated curve shown. The consolidated curve has to be drawn by removing the comments on some of the code加 d

commenting the ∞de出atis not needed. Since出eoptima1 solution cannot be c1early established，山巴zoomfeature ofMATLAB is used to narrow down the solution.

皐芯2.:.J....且ー (themain script file) も Chapter2: Optimizaヒionwith MATLAB も Dr. P.Venkataraman も Example2.3 Sec.2.3 者

も graphicalsolution u6ing MATLAB (two design variables)

苦 Optimaldesign of a Flag Pole for high winds

告 Ref. 2.4

者ーー'・ーーー・岨ー・ーーーー----・ーー・ーーー・ーー・ー司司・・ーーーー-----ーーーーー・・"

も globalstatement is used to share same information % between various m-files

global ELAS SIGALL TAUALL GAM FS GRAV

global RHO CD FLAGW SPEED LP L DELT

も-ー----------ーー・・・ーーーーーーーーー"・・ー-ーー・ーーー----ーーーーーーーーーーー"

も Initializevalues

ELAS 200e+09i も Modulusof elasticity -Pa

SIGALL = 250E+06i % allowab工enormal stress -Pa TAUALL = 145e+06i 告 allowableshear stress -Pa

lI li叩肺判明判明附問問削7岬榊糊州側刷叫州出印刷山品川III剛山市酔一一一1

68 GRAPHICAl OPTIMIZATION

GAM= 7860; FS 2.5; GRAV 9.81; 者ーーーーーー・ーーーーーーーーーーー-

RHO l. 225; CD l. 0; FLAGW 5000; SPEED 156.46; 者ーーー"ー同情ー・ーーーーーーーーー-

LP 8;

L 10; DELT 0.05; 告ー-ーー・ーーーーーーーーーーーー-

glval SIGALL/FS

g2val TAUALL/FS g3val DELT g4val 0.001

者 densityof material ・ kg/m3

% factor of safety 告 gravitationalacceleration

者 densityof air ・ kg/m3% drag coefficient 者 concentratedload on flag -N

老町I/S

者 locationof drag load on flag -m

も lengthof pole 者 allowabledeflection -m

も righthand side values

告 forthe constraints

者--喧・--ー・・ー・ーーーーーーー'ーーーーーーーーー司ー・ーーーーーーーーーーー-ーーーーー-----

x工田0.02:0.01:1; 者 the semi-colon at the end prevents 告 theecho x2=0.025:0.01:1; 告 theseare also the side consヒraintst x1 and x2 are vectors filled with numbers も notea way to avoid x1 x2 [X1 X2] meshgrid{x1，x2); 宅 generatesmatrices X1 and X2 corresponding to も vectorsx1 and x2

f1 obj_ex3(X1，X2); 者 theobjective function is evaluated over the entire

者 mesh

者 Constraintsare evaluated ineq1 ineq1_ex3{X1，X2); ineq2 ineq2_ex3(X1，X2); ineq3 ineq3_ex3{X1，X2); ineq4 ineq4_ex3(X1，X2);

[C1，han1] contour{xl，x2，f1， [0，10000，50000， 100000， 150000， 200000， 250000， 300000] ，'g-'); clabel(C1，han1); set(gca， 'xtick'， [0 0.2 0.4 0.6 0.8 l.O]) set(gca， 'ytick'， [0 0.2 0.4 0.6 0.8 1.0)) xlabel('outside diameter' ，'FontName' ，'times'，， FontSize' ，12) ;

E


ylabel{'inside diameter' ，'FontName' ，'times'，， FontSize' ，12) grid 告 holdon

figureも anew figure window contour{x1，x2，ineq1， [glval，glval] ，'r-'); hold on

も drawanother contour at 10者 theconstraint boundary contour{x1，x2，ineq1， [O.l*glval，O.l*glval]， 'b-'); set{gca， 'xtick'， [0 0.2 0.4 0.6 0.8 l.O]) set{gca， 'ytick'， [0 0.2 0.4 0.6 0.8 l.O])

xlabel{'outside diameter'，'FontName' ，'times'，' ... FontSize' ，12); ylabel{'inside diameter' ，'FontName' ，'times'，'. FontSize' ，12) ; hold off grid

も thefollowing code may be usefu1 for the consolidated 者 figure. Not used here __ uncomment below 者 [C2，han2] contour{x1，x2，ineq1， [glval， glval] ，'r-'); 者 clabel{C2，han2);者 set{h2，'Linewidth'，1) 苦 k2 gtext('gl');

者 set{k2，'FontName'，'Times'，'FontWeight'，'bold'，. 者 FontSize'，14奄，， Color' ，， red')

figure contour(x1，x2，ineq2， [g2val，g2val]，'r-'); hold on contour (x1，x2， ineq2， [0 .1*g2val， 0.1 *g2val] ，， b-' ) ; set{gca，'xtick'，[O 0.2 0.4 0.6 0.8 l.0)) set{gca， 'ytick'， [0 0.2 0.4 0.6 0.8 l.0)) xlabel{'outside diameter' ，'FontName'，'times'，， FontSize' ，12); ylabel{'inside diameter' ，'FontName'， 'times'， 'FontSize'，12); hold off

grid

者[C3，han3] contour(x1，x2，ineq2， [g2val，g2val]，'r--'); 者clabel{C3，han3);もset(h3，'LineWidth'，1)


者k3 gtext('g2');

者seヒ(k3，'FontName'，'Times' ，'FontWeight' ，'bold'，も， FontSize' ，14，'Color' ，'red')

figure

contour(x1，X2，ineq3， [g3val，g3val]，'r-'); hold on

contour(xl，x2，ineq3， [0.1*g3val，0.1*g3val] ，'b-'); set(gca， 'xtick'， [0 0.2 0.4 0.6 0.8 1.0]) set(gca， 'ytick'， [0 0.2 0.4 0.6 0.8 1.0])

xlabel('outside diameter' ，'FontName' ，'times'，， FontSize' ，12) ;

ylabel('inside diameter' ，'FontName'，'times'，， FontSize' ，12); hold off

~rid

者 [C4，han4] m contour(x1，x2，ineq3， [g3val，g3val] ，'b-'); 者 clabel(C4，han4);者 set(h4，'LineWidth'，1) 告 k4 = gtext('g3')i

もset(k4，'FontName'，'Times' ，'FontWeight' ，'bold'，も，FontSize' ，14，'Color' ，'blue')

figure

contour(x1，x2，ineq4， [g4val，g4val]， 'r-') i

hold on

contour(x1，x2，ineq4， [0.001*g4val，0.001*g4val] ，'b-')i set(gca， 'xtick'， [0 0.2 0.4 0.6 0.8 1.0]) set(gca，'ytick'， [0 0.2 0.4 0.6 0.8 1.0])

xlabel('outside diameter' ，'FontName' ，'times'，， FontSize' ，12) i

ylabel('inside diameter' ，'FontName'，'times'，， FontSize' ，12) ; hold off

grid

も [C5，han5] contour(xl，x2，ineq4， [g4val，g4val]，者，b--' ) i

も clabel(C5，han5)iも set(h5，'LineWidth'，1) 者 k5 gtext('g4')i

者 set(k5，'FontName'，'Times' ，'FontWeight' ，'bold'，者， FontSize' ，14，'Color'，'blue')

2.3 AOOITIONAL EXAMPLES 71

も theequality and in~quality constraints are も notwritten with 0 on the right hand side. もIfyou do write them that way you would have t。者 include [0，0] in the contour commands

1neal ex3.....m (the first constraint)

function retval ineq1_ex3(Xl，X2) 者 globalstatement is used to share same information ， between various m-files global ELAS SIGALL TAUALL GAM FS GRAV global RHO CD FLAGW SPEED LP L DELT

AREA 0.25* pi*(X1. 2 -X2.2)i 者 matrix

INERTIA = pi*(X1. 4 ・ X2. 4)!64i 者 matrix

FD 0.5*RHO*SPEED*SPEED*CD*X1i

MW 0.25*FD*L*L.*X1.!INERT1Ai

MF 0.5*FLAGW * LP*Xl.!INERT1Ai

SIGW GAM合GRAV*Li

retval MW + MF + SIGWi者 SIGWis added to al1 matrix

% elements

1nea2 ex3.m (the second constraint)

function retval ineq2_ex3(X1，X2) 者 globalstatement is used to share same information

者 betweenvarious m-files global ELA8 8IGALL TAUALL GAM FS GRAV global RHO CD FLAGW SPEED LP L DELT

AREA 0.25* pi*(X1. 2 -X2. 2)i

INERTIA ~ pi*(X1. 4 -X2. 4)!64i

FD 0.5*RHO*SPEED*SPEED*CD*X1; 8 FLAGW + (FD 合 L)i

o (X1.*X1 + X1.*X2 + X2.*X2)!6.0i

retval 8.*0./工NERTIA;

ineaヨ白1t:L.m (the third constraint)

function retval ineq3_ex3(X1，X2) 者 globalstatement is used to share same info幻nation

者 betweenvarious m-files global ELAS. S工GALLTAUALL GAM FS GRAV global RHO CD FLAGW SPEED LP L DELT

72 GRAPHICAL opnMIZATION

AREA 0.25* pi*(Xl. 2 -X2.2);

INERTIA pi*(Xl. 4 ・ X2.4)/64;

FD = 0.5*RHO*SPEED*SPEED*CD*Xl; dw FD*L 4./(8*E凶 S合INERTIA); df (2.0合FLAGW*L3 -FLAGW*L*L*LP) ./(ELAS*工NERTIA);

retval dw + df;

in自白4 ex3.m (the fourth constraint)

function retval ineq4_ex3(Xl，X2) retval Xl ・ X2;

Figure 2.9 displays the graphical or consolidated solution to由巳problcm.百lecodc

is available泊 lhem.file above (by commenting the codes白紙crealcthe new figure

windows and removing the comments on the codc白紙iscurrcntly commentcd). Thc optimal solution is not very c1ear. Figu問 2.10is obtain巴dby zooming in near出e

neighborhood of 0.6. Sec help zoom for in附 uctionon its use. lt can be achicved by

F1llg Polc戸。blcm

0.81-φ

』

a

o

a

婚

のu

n

u

』

叫

】

UES唱曲沼包-

0.2

0.2 0.4 0.6 0.8 outside diameler

Flgure 2.9 Graphlcal solulion: Example 2.3.

2.3 ADDlTIONAL EXAMPLES 73

，~

0.58 0.6 0.62 0.64 0.66 0.68 0.7 0.72 0.76 0.8 outside diamClcr

Figure 2.10 Graphical solutlon (zoomed):臥:ample2.3.

typing zoom at the workspace prompt and using the mouse to drag a rectangle around 曲eregion that n民 dsto be enlarged. From Figure 2.10出csolution is around the outsidc diameter of 0.68 m胡 dtllc inside diameter of 0.65 m. Typing zoom in出巴

workspace again will toggle白白伊回 backto nonnaJ state. In Figure 2.10，山tickmarks are placed on the figure through出ecommand window for bcttcr inte中肉tationof the solution.

The graphics in由eabove ex副nplewere created using the same statemenls

encountered previously. Color contours were used to establish the feasible region.τhe zoom feature was employ吋 toobtain a bettcr estimatc of the solution.

2.3.3 Example 2.4

This example is合um出earca of heat transfer. The problem is to design a triangular fin of白巴 smallestvolume that will at least deliver specificd fin efficiencies. The

graphical ti回 tu陀 ofthis code is very similar to Example 2.3. ln山iscxample， the inequality constraints are computed and retumed from a single function m.file rather

山anseparatc files considered in the previous example. Another ncw fealure in this example is to invoke special mathematical functions， the sesseI functions that a問

74 GRAPHICAL OPTlMIZATION

available in MATLAB.百世sexample also iI1ustr官民sa problem where the optimization

problem can be adequately defined but the solution is easily determined from the

equality and side cons位aints.In other words， the problem can easily accommodate additional demanding constraints. In large complcx mathematical models with many

dcsign variablcs it is not easy to ensure that at least one of thc inequality constr瓜ntsIS active.

Des勿'nProblem: M凶凶ze由巳 amountof material used in出edωi伊 ofascrics

of identical triangular fins that cover a givcn area and operate at or abovc the specified cfficicnciω.

Mathematlcal Model: This type of problem should be able to accommodate

several dcsign variables. In this section，出eproblem is set up to have two design

variabl邸.百1efin material is aluminum.

Design Parameters: For aluminum

h = 50 W/m2 [convection coefficicnt]

k = 177 W/m-K [thermal conductivity) N = 20 [number of fins] W=O.1 m [wid山ofthefins)

Fin gap is th巴samcas the base length of the triangular fin.

Design Variables

b: base of the triangular fin L: height of thc triangular fin

Figurc 2.11 iJlus佐atω 山egeom巴町yof自己fins.

Geometric Relatlons: Th巴followingare some of th巴areacalculations that are uscd

later in the dcvelopmcnt of山ccons佐aints:

Af= (2N -1)*b叫V footprint of出efin and gap Ac = 2*W*[L2 + (b!2)2]112 fin area for heat transfer

Ab = (N -l)*b*W gap ar伺

At = N*Ac + Ab total area for heat transfer

Objective Function: The total volume of ma回 ialfor the fin is

j(b， L) = O.5*N*W*b*L (2.16)


b b

FJgure 2.11 Fln deslgn for heat transfer: Example 2.4.

Constraint Funct，幻ns:Thc heat transfer equations for the fin arc available in Reference 7. Fins are typically mounted on a specified 紅白 . 明1efirst cons回 intis one

on area:

h(b， L): Af = 0.015 m2 (2.17)

Note:山isconstraint essentially fixes the value of b. In山egraph this constraint wil¥ be a straight line parallel to出cLaxis.lf血iswcre to happen in a problcm with scveraI design variables， it would be prudent to eliminate出isvariablc from the mathematical model by identifying it as a design par佃 leter.

百 eefficiency of a single fin can be establishedぉ

ηr = (l/mL) 1，(2mL)lIo(2mL) where

m= (2h/，肋 )112

叩 d出el' s are Bessel equations of the first kind. The first incquality constraint is

g，(b， L): ηf~ 0.95

The overall cfficiency for the heat transfer is

TJo= l-N*叫ん41)(ト叩

The final inequality constraint is

g2(b， L): TJo ~ 0.94

Side Constraints: The side cons住aints釘芭

0.001 SbSO.∞5

0.01 S L S 0.03

(2.18)

(2.19)

(2.20)


MATLAB Code: Similar to由eprevious example. the hash marks on the inequality cons佐aintcan be established by plotting two vaIues of the constraint: the limit value and a value higher. Once出巴yhav巴beendetermined. then the consolidated figure can be established. Th巴codeshown here will draw aIl the functions in separate windows. For the consolidated plot this code necds to be edited. It is left to the student錨 m

exerclse.

盟主ムmThe main script file

もChapter2: Optimization with MATLAB % Dr. P.Venkataraman % Example 2.4 Sec.2.3 者

者 graphicalsolution using MATLAB (two design variables) 者 Minimumfin volume for efficient heat transfer も materialAluminum 者ーーーーーー-----------・・・・・------ーーーーーー・ーーーーーーー--ーーーーーーーー

者 globalstatement is used to share same information 者 betweenvarious m-files global N H K W AREA 者・"・----ーーー司--ーーーーー-ーーーーーーーーーーーーーーーーーーー一一ーーー・・ーー回・ー--・

も工nitializevalues N 20 告 numberof fins W 0.1 者 widthof fins 日 50.0 者 convectioncoefficient W/m*m K 177.0者 thermalconductivity W/m-K AREA 0.015者 availablefin foot print area 者・・ーーーーーーーーーーーーーーーーーーーーーーーーーー・・ーーーーー-------ーーーーーー・ーーー・

者 righthand limits for the functions h1val AREA; glval 0.95; g2val 0.94; もーーーーーーーー---ーーーーーー・ー・ーー・・ー・・ーー・・四ーー・ーー・・ーーーーーー・・ー・『ーー-

x1=0.001:0.0001:0.005; x2=0.01:0.001:0.03; も x1and x2 are vectors filled with 者 numbers[X1 X2] meshgrid(x1，x2) ; も generatesmatrices X1 and X2 corresponding to も vectorsx1 and x2

f1 = obj_ex4(X1，X2); も theobjective function is evaluated over the entire 者 mesh

も Constraintsare evaluated

!配

，I't'

2.3 ADDITIONAL EXAMPLES n

eq1 eq1_ex4(X1，X2); 苦 theequality constraint

[ineq1， ineq2] ine~ex4(X1 ， X2);

者 inthe above note that the two inequality constraints 者 areobtained simultaneously. Keep in mind も thateach constraint is a matrix

者 readyfor the plots

[C1，han1] contour(x1，x2，f1， [0.00001， 0.00002，... 0.00004， 0.00006， 0.00008， O.OOOl]，'g・');

clabel(Cl，han1) ;

set (gca，， xtick' ， [0.001 0.0015 0.002 0.0025 0.003 0.0035 0.004 0.0045 0.005])

set(gca， 'ytick'， [0.010 0.015 0.02 0.025 0.03 0.125]); xlabel('fin length' ，'FontName' ，'times' ，'FontSize' ，12); yl油 el('finwidth' ，'FontName'， 'times'， 'FontSize'，12) grid % hold on

figureも opena new figure window も p10ttingthe equality constraint contour(x1，x2，eq1， [h1val，h1val]，'r-'); set (gca，'xtick'， [0.001 0.0015 0.002 0.0025 0.003

0.0035 0.004 0.0045 0.005])

set(gca， 'ytick'， [0.010 0.015 0..02 0.025 0.03 0.125]);

xlabel('fin length' ，'FontName'， 'times'， 'FontSize'，12); ylabel('fin width' ，'FontName' ，'times' ，'FontSize' ，12) grid

も plottingthe first inequality constraint figure

contour (x1， x2 ， ineq1， [0. 968 ， 0 . 968] ，， r -， ) ; hold on

contour(x1，x2，ineq1， [0.969，0.969] ，'b-'); set(gca，'xtick'， [0.001 0.0015 0.002 0.0025 0.003

0.0035 0.004 0.0045 0.005])

set(gca， 'ytick'， [0.010 0.015 0.02 0.025 0.03 0.125J); xlabel('fin length' ，'FontName' ，'times' ，'FontSize' ，12); ylabel (' fin width' ，， FontNar陪，， 'times' ，， FontSize' ，12) hold off grid

者 plottingthe second inequality constraint


figure contour(x1，x2，ineq2， [g2val，g2val]，'r-'); hold on

contour(x1，x2，ineq2， [1.01*g2val，1.01*g2val] ，'b-'); set(gca，'xtick'，[0.001 0.0015 0.002 0.0025 0.003

0.0035 0.004 0.0045 0.005]) set(gca，'ytick'， [0.010 0.015 0.02 0.025 0.03 0.125]); xlabel('fin length' ，'FontName'，'times'，'FontSize'，12); ylabel('fin width' ，'FontName' ，'times' ，'FontSize'，12) hold off grid

nb.Le組担割leobjective function

function retval obj_ex4(X1，X2) 者 volumeof the fin global N H K W AREA retval 0.5*N*W*X1.*X2

量qlJ雄.mThe equality ∞田佐直int

function retval eql_ex4(Xl，X2) も theequality constraint on area global N H K W AREA retval (2.0合N -1)*W*Xl;

旭叫~The inequ~lity constraints

function [retl， ret2] ine~ex4(Xl ， X2)

者 returnsboth the inequality constraints global N H K W AREA c 2*sqrt(2.0)*sqrt(日/K)*X2./sqrt (Xl) ; retl (besseli(1，c)../( (0.5*c) .*besseli(O，c))); Ae 2.0*W*sqrt((X2.*X2 + .25*Xl.*X1)); Ab (N-1)*W*X1; At N*Ae + Ab;

Ar Af ./At; ret2 (1.0 -N *Ar.合 (1 -ret1));

The above script files should display each function in a scparate figure window. The inequality constnlint figures should contain two contours with山cblue contour indicating白efeasible region. Fig山な 2.12displays a composite plot The solution is detcnnined by the equality constraint and the lower Iimit on白efinleng白ぉdiscussedpreviously.

1m

1量産l

E

..，

，11<'

，."，'

2.4 AoomONAl MATl.AD GAAPHICS

0.045

iO悶

0.025

1.5 2 2.5 3 3.5 fin base(m)

円gure2.12 Graphlcal solulion: Example 2.4.

2.4 ADDITIONAL MATLAB GRAPHICS

79

S

x10-3

This section providωa brief exposure to additional features of Handle Graphics， inc1uding development of a GUI in MATLAB. The coverage is very modest and is inc1uded hcre for thc sake of complcteness as we have accomplishcd our goal of graphical optimization泊山eprevious sections. This section can be avoided， postponed， or emphasized for the projects in也ecourse， as it has no additional information for graphical optimization. It does contain reasonablc infonnation in undcrstanding MATLAB graphics and programming. The reader is encouraged to take somc time getting familiar with出eversion ofMATI.AB installed on his machine. There are usually significant enhancements in newcr versions of MATLAB. For example， MATLABV巴rsion5.3(出isbook is based on Version 5.2) allows you to customize your plot and add annotations to it白roughadditiona1 editing ti回 1町民of由巳自仰向window.

You do not have to customize your figure through code alone. In this section， we will a1so be developing the GUI using MATLAB Handle Graphics so白紙 wehave more

experien白 wi血MATI.ABHandle Graphics. MATI.AB doωprovidc a GUI creation tool， guide，出atthe user is recommended to explore.

In the previous section there were only two design variables. An cq回lIyspaced vector was obtained for each variablc based on由巳 sidcconstraint. A m巴shgridwas

80 GRAPHICAL OPTIMIZA TION

出engenerated. AlI functions werc evaluated on this grid. Contour plo15 were drawn

for the functions representing出巴 objectiveand constraints. Except for the objective

function which displayed several contours， the constraints were only drawn for a

single valuc. This procedure suggests the possibility of automation. User input will be necessary for selccting出巴悶ngcfor the variables as well as thc functions出atwill be plotted.

In this scction， we will create plotting facility to include a GUI that will

• Obtain白enumber of plo15

. The range and incrcment of the design variables

• Use a file selection box/utility出atthe user will us巴toidentify出efunction to

be plotted

• Prompt the user for the number of contour百foreach plot

・Allowselection of contour levels

・Creatcthe plot

百leexcrcise will requirc two m-files: one 10 set up出eGUIand山eo白巴rto assist in

the plotting. The user will be able to tweak thc plot in the Command window after it is drawn. The following subsections pωsent a brief introduction to Handlc Graphics

and GUI controls in Version 5.3.

2.4.1 Handle Graphics

MATI..AB Handle Graphics refer to a collection of low-level graphic routines that actually generatc or carry out graphical changes you s田 inthcMA百.ABfigure window.

For the most part these routines are transparent阻 dtypically users nced not be awarc

of them. In fact， to understand optimization and to program the techniques in MATI..AB， it is not necessary to know about Handle Graphics since using the higher-level plot

commands such as plot and contour is sufficient to get the job done."Thesc commands actually kick in scvcral of白eHandle Graphics routines 10 display白eplot on the figure. A notewor註lyfeature of MATI..AB graphics is白紙 itis implemented泊佃

object圃 orientedmanncr. What this means is that most of the 印刷esyou see on the figurc， Iike axes， labels， tcxt， and lines， arc all objec15. This mcans that most of the 蜘

graphical items on the figure have properties白紙 C組 bechanged出roughprogram

code. In the previous sections we used this to manipulate color， font size， lin巴wid白，and the Iike.

In order to change出eprope凶esof objects， it is nec凶 saryto identify or refer to

thcm. Objects in MATLAB are identified by a unique number referredωas a handle. Handles can be assigned to all objec15 when they arc crcatcd. Some objec15紅巴partof

other objccts. Objects that contain other objec15 arc referred to as container or parent 利回民.There is a definitc hierarchi叫 s回 ctureamong objects.τ'he root of all

MATLAB graphic 0切回15is出efigure obj田t.The figure 0同ectfor example can hold

sevcral GUI controls， which are known as uicontrol objec15. It 31so contains the蹴 es

2.4 AOOITIONAL MAT¥AB GRAPHICS 81

obj田t.百巴回凶 objωtin tum has the line， text， and image object. All 0対ectsin

MATLAB can be associated wi白aset of properties白atis usually based on i15 function

or usefulncss. Different object types have different sets of properties. Each of thcs巴

prope凶esis describcd through a pair of related information (name， value).百 efirst

element of this pair is the namc for the property and the second is the value

corresponding to恥 property.When出eo同ectis created， it inherits all of the default

prope凶es.For example:

plot(ヒ， sin(2*ヒ)，'-go'，， LineWidヒh'，2，， MarkerEdgeColor' ，'k'，， MarkerFaceColor' ， [0.49 1 0.63]，， MarkerSize' ，12，)

The last four 1ines abovc represent four properties of出eplotted line that is being

changed. The property narn凶 on山eIcft can be easily understood.百leformat size

units are in points. The color value is a row vcctor of出向巴colorvalues between zero

and onc representing the red， green， and blue values， respectively. There are sevcral oth巴rprope而esof the plotted line which will be sct at thcir default values. Thc

MATLAB online rcference should Iist all objec15 and thcir prope凶es.In general. object properties clln be changed by using the ser function to change

their values. Similarly， object properties c佃 bercad by the use of the get function. To inquire about the property of an 0附田t:

get(handle，'Property Name') geヒ(h_1，'Color')

In the above， h_l is出ehandle of the object whose color is desircd. Similarly，

seヒ(handle，'Property Name'， 'Property Value') set(h_l，'Color' ，'r') or set(h_1，'Color'， [1 0 0])

in the above出巴 valueof Color for the objcct represcnt叫 bythe handlc h_l is set 10

rcd in two ways. The sccond specifiωthe color red through the vector valuc. It is expectcd that the object identified through h_1 should have a Color property. MATt.AB

will inform you of thc error in case you are assigning a property/value伽 tis not valid

for the object.

2.4.2 Graphlcal User Interface

The most significant featurc of software design and development today is the

object-oriented design paradigm白紙 wasbriefly iIIustrated in出eprevious section.

The combination of the above with event-driven programming is both a natural and a

陀 quiredfcaturc in current softwarc programs. Instead of the programs controlled by


a predetermined processing s叫uence(batch processing)， today's processing is govemed by the user through some dirl田 tedmouse clicks and buttons. These user

selections cause certain events that instruct the software to carry out some action. The

o句民俗伽1fire血eevenωare usually referred 10ぉ userinte巾叩 elcmc附.It is

important to understand that thc same evcnts can be triggered by obtaining user

responsc to sc隠 enprompts al the command linc (unattractivc). Using buttons to

achieve the same result i5 more glamorous although it uses more compu旬tion叫

陀 sources.ln fact， for commercial success it is n配 essaryto include this mechanism of

user inte.raction in the software product. Also. users expect these elements 10 behave in a standard way. There are many elements白紙 theuser can use to interact with

program5. Many applications across several disciplines havc now defined a minimum

5tandard collection of出eseelements. MATLAB必50providcs them. The organization

of山eseelemenls and their presentation to the user in a graphlcal manner is identified

as出egrョphlcaluser interfa田.A number of GUI-based t∞1s.釦nctions.催芭 available

at the MATLAB site.

This section provides only a brief introduction to出eelements used in the example.

Th巴seelcments are primarily used through MATLAB Handlc Graphics programming.

Consider the casc of a button白atthe user can push to cause some action. referred to

as a push button in thc subsequent discussion. At the outset. there ru官 twoprincipal

requiremcnts山atare cxpectl叫 ofthe push button. First， the凶 ermust be able to see

it. Second. whcn出euser clicks on it (single versus double clicks can be distinguished in many software development codes) the button must飽 spondwi出 someexpected

action. In MA TLAB the push button is a graphical ot羽田t.It has many prope両国thatcan

be used to describe its app回 ranceon thc screcn. In MA'τLAB. all graphical objects can onlyappe釘 ina figurc window. Hencc， a figure window is essential to display thc

push button.百 isfigure window actsωa container and can contain many user

interface elcmcnts. Considering the presentation of the button on thc scrccn， it is

ne∞ssary to dccide where the button should app回 ron出efigw沼window，how big it should be， what label it should have (typically indicates血巴kindof action it is likely

ωcause)，加dso on. Sincc the push button is a graphical object， much of these

prope凶esc叩 beset if therc is a handle available for Ihe push-button objcct. Once the graphl伺 1appearance is tak

Hpush uicontrol(Hfig_1，'Style'， 'push'， 'Units' ，'inches'，

E苦J

自

11

2.4 ADDITIONAL MATLAB GRAPHICS 83

'Position'， [2.5，0.1，0.7， .2]， 'BackgroundColor'，'red' ，， String' ，， EXIT' ， 'Callback' ，'close');

In the abovc Iines of codc. Hpush is the handle to the push-button object. It is created

by using the function uicontrol. This is the function that will create the user intcrface

elements. The same function is used to place several di庄erentuser interface elements

through the selection of the style property. In the example below. these include label

boxcs and user editable texl boxes. Several parameters identify Lhe typ巴of巴lemcnt

UEat is created and the properties UEat need to beset.Each propeny is set through a

propertynamelpropertyvalue pair. Hfiιlin恥Iincsof code abovc児島内 tothe

handle of the figure window in which this element wiU be placed. The type of element created is a push-button element cstablish吋 through恥 Styleproperty s凶ngname

‘push." The Units used to size and position the push button is inches. The Position

property is a row vector of fo町 valuesgiving the Left. Bottom. Width. Height ofthe

element in the Units sclected earlier. The default location of the origin is the bottom

lcft comer of the figu問 window.The BackgroundColor for the button is sct to red. Thc

label that wil¥ be on the face of the push button is EXIT. This is the TCXl property of the push-button object. Thc CallBack property of the push button is白estring “close." This string will be thc parameter in the eval function出atis called when出巴pushlclick

event is trigge陀 d.MATLAB command close will be executed in the Command

workspace when the push button is p問 ssed.Thls will cause lhe figure window displaying the push button to c1osc.

Only a few elements are used in出isscction. The Cal1Back strings are also quite s回 ighばorw副.Before dcveloping thc code. it is useful to observe恥 hlerarchical

order in which the graphical objects in MATLAD are organized (reproduced from血e

MATLAB documentation) in Figure 2.13. The root implies the starting resource for aIl graphic displays in MATLAB. Its handle usually has a value of O. The root also serves as a container for all of the figure windows. Another exprcssion for山isrelationship

is that a figure window is the child of the root. From the hierarchy it is essential ωnote 白紙出euser interface elements. uiconヒrolor uimenu， c叩 onlyexist wi出ina

Ro+ ot

Ulcontrol Axes Ulmenu

Image Llne Patch Surface Text light

F1gure 2.13 Hlerarchical slruclure lor MATlAB graphlcal objecls.

84 GRAPHICAL OPTIMlZAnON

figure window. 80 al50 the蹴 e5object. In the following code each interface element is first associated wi白ahandle 50出創出eproperty of the element can be陀 fen吋 toif necessary. This is especially true of the text entry boxes to invok巴theCall8ack s凶ng5.

2.4.3 GUI Code

百lefollowing code wil1 generate the GUI. The uiwait 陀quiresthal山iswindow be closed before subsequent processing can take place (or uiresume be uSed in白E

∞de) .

.Gll旦..yl

も GUIfor Plotting facility も Dど. P.Venkataraman Applied Optimization Using MATLAB 者

% Ch.2 Sec. 2.4 若

者 GUIis created within a figure window 告ーーー------ーーーーーーーーーーーーーーーー-ーーーー司ーーーーーーー・・・・ーーーーーーーーーー

Hfig_1 '" figure; 者 Hfig_1is the handle to the も figurewindow

set(Hfig_l， 'Color'， [0.3，0.6，0.5]， ...も setcolor ， NumberTitle' ， 'off'，告 nowindow title ， Name' ， 'Set Range for Design variables'， 'Units' ，'inches'，者 inchesused to layout % other controls 'Menubar' ， 'none' ， 'Position'， [4，4，4.4，1.5]);

告Positionproperty is [left， bottom， width， height] in 者 Unitsselected

Ht_1 uicontrol(Hfig_1，， Style' ，， text' ， 'Units'， 'inches'， 'position'， [0.1，1.1，2.4， .2]， 'String'，'Number of Functions to Ploヒ， ) ;

]

司，也氏J

'

・

nu

--，

-・・

1

qd

・

・1

・'l

s

L

'

'

H

'

8

6

f

、，

e・

可ムい」

hU

弓L

o--CI

r・Gn'

te--'

n''n

o''o

c''・1

・1es↑』

ulti

yis

ztno

sup

円

4'''

t

H

2.4 ADOITIONAL MATLAD GRAPHICS 85

， BackgroundColor' ，'white'，， String' ， ' ，

'Callback'，'Np str2nUm(get(Ht_2，"String"))i');

HL X1L uicontrol(Hfig_l， 'Style'，'text' ， 'Units' ，'inches'， 'Position'， [0.1，0.8，0.5， .2]， 'String' ，'x1(min)')i

HT_XIL uicontrol(Hfig_l， 'Style' ，'edit'， 'Units' ，'inches'， 'Position'， [0.7，0.8，0.7， .2]， 'BackgroundColor' ，'white'， 'String'，" ， 'Callback' ，'xlmin str2num(get(HT XIL，'

， String' ， )) i') j ー

HL_XIM uicontrol(Hfig_1， 'Style'，'text'， 'Units'， 'inches'， 'Position'， [1.5，0.8，0.6， .2]， 'String'， 'incremenヒ')i

HT X1M uicontrol(Hfig_l， 'Style' ，'edit'， 'Units' ，'inches'， 'Position'， [2.2，0.8，0.7， .2]，， BackgroundColor' ，'white'，， String' ，，， 'Callback' ，'x1inc str2num(get(HT_X1M，'

， String' ， )) j') j

HL_X1U uicontrol(Hfig_1，， Style' ，， text' ， 'Units' ，'inches' ， 'Position'， (3.0，0.8，0.5， .2]， 'String'，'xl(max)')j

HT_X1U uicontrol(Hfig_l， 'Style'， 'edit'， 'Units' ，'inches'， 'Position'， (3.6，0.8，0.7， .2]，

86 GRAPHICAlOPTIMIZATION

， BackgroundColor' ，'white'，， String' ，，， 'Callbacl屯，，'x1max str2num(geヒ(HT_X1U，，

， String' ， ) ) ; ， ) ;

E江，_X2L uicontrol (Hfig_l，

， Style' ，， text' ， 'Units' ，'inches'， 'Position'， [0.1，0.5，0.5， .2)， 'string'，'x2(min)');

HT_X2L = uicontrol(Hfig_l，， Style' ，， edit' ， 'units' ，'inches'， 'Position'， [0.7，0.5，0.7， .2)， 'BackgroundColor'，'white' ，， String' ，，， 'Callback' ，'x2min str2num(get(HT_X2L，'

， String' ， ) ) ; ， ) ;

HL_X2M uicontrol(Hfi9_1，， Style' ，， text' ， 'Units' ，'inches'， 'Position'， [1.5，0.5，0.6， .2)， 'String' ，'increment');

HT_X2M uicontrol(Hfig_1，， Style' ，'edit' ， 'Units' ，'inches'， 'position'， [2.2，0.5，0.7，.2)， 'BackgroundColor'，'white' ， 'String' ， ' ， 'Callback' ，'x2inc str2num(get(HT_X2M，'

， String' ，) ) ;') ;

HL_X2U uicontrol(Hfig_1， 'Style'， 'text'， 'Units' ，'inches'， 'Position'， [3.0，0.5，0.5，.2]， 'string' ，'x2(max)');

HT_X2U uicontrol(Hfig_1， 'Style'， 'edit'， 'Units' ，'inches'，

2.4 ADDITIONAl MA11AIi GRAPHICS 87

'Position'， [3.6，0.5，0.7， .2]， 'BackgroundColor'，'white'，， String' ， ' ，， Callbacl屯，，'x2max str2num(get(HT_X2U，'

， String' ， ) ) ; ， ) ;

Hlabel uicontrol(Hfig_1，， Style' ，， text' ， 'Units'， 'inches'， 'position'， [0.1，0.1，2.2， .21， 'BackgroundColor' ，'y'， 'String' ，'After entering values please press EX工T') ;

Hpush uicontrol(Hfig_1，'Style' ，'push'， 'units'， 'inches'， 'Position'， [2.5，0.1，0.7， .21，， BackgroundColor' ，'red'，， String' ， 'EXIT' ， 'Callback' ，'close');

uiwait

In the above code出estatemenlS should be cωY to interpret. Much of the code c叩 beedit凶 aftercopy and p邸 teoperation. The last command ui wai t instruclS MATLAB to hold off execution until the user closes出eGUI window. Figu陀 2.14shows the image of the GUIぉ itapp回 rsin MATLAB.百letext boxes arc initializ.ed with no values forcing血euser to set values出erebycausing al1 of the CallBack strings to be evaluated. In addition， th巴dataentered by the user is a s甘ing.It has to be converted to a number. The ∞dehe印 isbasic. It does not verify也atthe user did enter a number. 百 isverification can be done by extending血eCallBack string over multiple statemenlS. This is leftぉ anexercise in programming for the user.

The script m-file used to select佃 dcreate the pl0lS is as follows.

目盛m.m

者 Scriptfile to go with Contour plotting 宅 Dr.P.Venkataraman， Applied Optimization Using MATLAB

-・・・・幽画圏直幽画趣醤岨血畠圃・・・・亡コ圃圃 E二コl1li圃 E二コ匝圃仁二コ園圃仁二コ圃圃仁二コ固圃仁二コ

関宮島睡眠融担醐暢腫酷謹繭置璽置EFlgure 2.14 Image of GUI.

88 GRAPHICAl OPTIMIZATlON

も Ch. 2， Sec 2.4 も

も scriptM fi1e for 20 Contour P10tting of functions % invo1ved in optimization 者 A11functions are expected to be avai1ab1e as 者 MATLABfunction m-fi1es 昔 Functionswil1 be se1ected through the input se1ection も box告Functioninput parameters wi11 be meshed matrices 告 basedon the range of parameters avai1ab1e from GU工も

も TheNumber of p10ts and range for the p10t も areobtained from running the 告 userinterface script GUI2_4 prior 者 tothe ca11 to P1ot20.m 者

GU工24 も callthe GUI

告 X工nfo，Ylnfo are introduced for introducing default 者 valuesXinfo [xlmin x1max x1inc1 ; も theva1ues x1min， x1max， x1inc are avai1ab1e in 者 theCommand workspace due to Ca11Back

if isempty(Xinfo) Xinfo [-4 4 0.051 告 defau1t

end fprintf ('¥n')

Yinfo [x2min x2max x2inc1 ; if isempty(Yinfo)

Yinfo ・44 0.051 end fprintf ('¥n' ) xvar Xinfo(1) :Xinfo(3) :xinfo(2); yvar Yinfo(1) :Yinfo(3) :Yinfo(2); [X1， X21= meshgrid(xvar，yvar);

% xl vector % x2 vector も matrixmesh

も setdefau1t number of p10ts to 1 if isempty(Np)

Np 1; end c1王;奄 c1earfigure

rl1:

2

• 1

1

百

2.4 AODITIONAl MA11AB GRAPHICS 89

for Mp10t l:Np

text1 ['The function which is being p10tヒedmust '¥nbe a MATLAB'

'¥nfunction M -Fi1e. Given a meshed matrix input' '¥nit must return a Matrix'

'¥nP1ease se1ect function name in the dia10g box . and hit return¥n ' 1 ;

% this prints the text to the screen in the command も windowand serves as a prompt 者 the¥n is a new 1ine command

fprintf(texヒ1)

も usingthe uigetfi1e dia10g box

[fi1e， pathl uigetfi1e('c:¥*.m' ，'Fi1es of type MATLAB m-fi1e'，300，300);

も checkif fi1e is string

者 stripthe .m extension from the fi1e so it can be 者 ca11edby the program

if isstr (file) functname strrep(file，' .m'，");

e1se fprintf ('¥n¥n' )

text2 [， You have chosen CANCEL or the fi1e was '¥nnot accepヒab1e'

'¥nThe program needs a Fi1e to Continue' '¥nP1ease ca11 P1ot20 again and choose a fi1e OR ' '¥npress the up-arrow button to scro11 through '¥nprevious commands ¥n¥n' ， Bye 1'];

error (text2) ; end

c1ear text1 text2; 告 c1earsthe variab1es text1 . も andtext2 for reuse

c1ear Fun maxva1 minva1 strcon conva1ue oneva1ue 1abcont 1abcontU Fun feva1(functname，Xl，X2); maxva1 max(max(Fun)); minva1 min(min(Fun));

fprintf (' The contour ranges from MIN: 昔12.3fMAX.も12.3f ' ，minva1，maxva1);

fprintf ('¥n' ) ;

strcon input('DO you want to set contour va1ues ?

90 GRAPHICAlopnMIZATION

[ no]:'，'S'); strconU upper(strcon) ; if strcmp(strconU，'YES') I strcmp(strco凶， 'Y') fprintf(' Input a vector of contour levels ，);

fprintf ('¥n' ) fprintf('between者10.2fand巷10.2f' ，minval，maxval); fprintf ('¥n' ) convalue input (' Input contour level as a Vector :'); labcont input('Oo you want labelled contours ?

[ no] : ' ，， s' ) ;

else

labcontU upper(labcont); if strcmp(labcontU，'YES') I strcmp(labcontU，'Y') [C，h] contour(xvar，yvar，Fun，convalue); clabel(C，h);

else contour(xvar，yvar，Fun，convalue);

end

ncon input('Input number of contours [20] :'); if isempty(ncon) ncon 20; labcont input('Oo you want labelled contours ?

[ no] : ' ，， s' ) labcontU upper(labcont); if strcmp(labcontU，'YES') I strcmp(labcontU，'Y')

[C，h] contour(xvar，yvar，Fun，ncon); clabel(C，h);

else contour(xvar，yvar，Fun，ncon);

end elseif ncon 1

onevalue input (' Input the single contour level ' ) ; labcont input (' 00 you want labelled contours ?

[ no] : ' ，， s' ) labcontU upper{labcont) ;

if strcmp(labcontU， 'YES') I strcmp(labcontU，'Y') [C，h] = contour(xvar， yvar， Fun， [onevalue，onevalue]);

clabel (C， h) ; else

end else

contour(xvar，yvar，Fun， [onevalue，onevalue]);

REFERENCES 91

labcont input ( ， 00 you want labelled contours ? [...

no] :' ，'s'); labcontU upper(labcont) ; if strcmp(labcontU，'YES') I strc即 (labcontU，， Y' )

else

end end

[C，h) contour (xvar， yvar， Fun，ncon) ; clabel(C，h);

contour(xvar，yvar，F山 l，ncon)end

if Np > 1 hold on;

end Hf gcf;

end

figure(Hf)i grid hold off

In the above the plotting commands have been used before. The new MATLAB commands are isempty， clf， isstr， strrep， max， min， upper， strcmp， and fprint. Use the help command 10 learn more about these. Run the script file叩 dunderstand the s叫uenceof actions as well俗曲eprompts regarding the contour. In 白isexample the functions白紙crealethe plot do not need global values for calculation 釘佃sfeπ'edfrom白ecommand workspace. In other words the functions白紙areto be plotted can be calculated independently.

REFERENCES

1. MATLAB The Langooge of Tecll1lical Compluing: Usillg MATLAB， Version 5， Ma山Wo北sInc.， 1998.

2. MATLAB The Language ofTechnical CO岬 uling:Using MATLAB Graphics， Version 5，

MathWorks Inc.， 1996.

3. Bohachevsky， 1. 0.， 10hn80n， M. E.， and Slein， M. L.， Generalized Simulated Annealing for FuncLion Optimization， TeclmometricムVol.28， No. 3，1986.

4. Arora， J. S.，lntroduclion 10 Oplilllum Design， McGraw-HiII， New York， 1989.

5. Be侃附阜釘rれ'，F.P.，andJゐ0加s蜘ω目，E.R.可.，JrYor泳k，1992.

6. Fox， R. W.， and McDonald， A. T.，lntroduclion 10 Fluid Mechanics， 4由ed.，Wiley， New York，1992.

7. Arpaci， V. S.， Conductioll Heat TrallSfer， Addison-Wesley， Reading， MA， 1966.


8. Hanselman， D.， and Littletield， B.， Mastering Matlab 5， A Comprehensive Tutorial and Reference.泊施 MA'τ'LABCurriculum Series， Prentice-HaJl， Englewood Cliffs. NY. 1996.

9. Building GUIs wi山MATLAB.online reference in pdf format. Math Works Inc.. 1998.

PROBLEMS

(Many of the graphica1 enhancements are now possible through plotedit features in la包rreleases of MA1LAB. The fo¥lowing problems are just suggestions. The suggested problems use additional graphical elements. and a1so r叫uiredeleting handles. Pleas巴

consult MATI..AB documentation.)

2.1 Plot the ba1lis即位ajectory合omsimple two-dimensiona1 m田 hanics.

2.2 Produc巴ananimated display of t11巴館司jectory.

2.3 Create a two-dimensional program that will display a random firing location and a random target location. Allow the user to choose initia1 velocity magnitude and angle. Plot his tr吋ectoryand ca1culate his error.

2.4 Create a thre巴・dimensionalprogram出atwill display a random firing location and a random target location. Allow出euser to choose initia1 velocity magnitude and direction angles. Plot his trajectory and calculate his error.

2.5 Draw the boundary layer profile of laminar l10w over a l1at plate. Draw lines indicating velocity profile at ten points in the boundary layer.

2.6 Create a program出atwill plot白巴 velocityprofile for user-specific inputs.

AlIow the user to express his inputs through a dia10g box.

3

LINEAR PROGRAMMING

The major part of this book deals with mnthemntica1 models血ntnre chnrncterized by

nonlinear equations. Iike most of由巳 examplesused for iIIustration白usfar. The presence of a single nonlinear equation in the model is sufficient to identify the

problem from the class of nonlinear programming (NLP) problems. Mathematical programming is ano出erterm that is used to describe such models and白eirsolution

飽chniques.Most of engine巴ringdesign falls in this category. Having represented such problems graphica11y in the previous chapter. it app儲 rs白紙 thegradient and th巴

C町 va胞reof血巴白nctionsin the mathematical model had a significant impact on

identifying血esolution， even though no effort was made to point out白isfeature sp即 ifically.In fact， graphical solution was obtained by inspection rather than determined by mathematical relations or entities.

There is an equally impo民組tcIass of problems whos巴mathematicalmodel is made up excIusively of functions that are only Iinear， such as Example 1.3，鎚 modeledin Equ姐 ons1.21-1.24. The same example wぉ modifiedin Equation5 1.25-1.28 50伽 t

a graphical solution could be di5cusSed. It is app釘ent合om血巴 graphica1description in Figure 1.6白at出町巴 isno curvature evident in白efigure. These problems are

旬m巴dlinear programming (LP) problems. They are natural in出esubject of operations research， which covers a vast variety of models used for several kinds of

decision making. Ex創nple1.3 does represent a decision-making problem. Typically， LP can be a course by itself (or several courses for伽 tmatter). A significant portion

of such a co町民 wouldbe to develop mathematica1 models from different areas of applications， as there is usua11y one numerical technique白atis commonly used to

solve LP problems. It is cal1ed the simplex method and is based on the algorithm by D佃包ig[1]. The method involves mostly elementary row operations typical1y

encountered in Gauss-elimination type m巴白odsthat are part of the numerical 93

94 LlNEAR PROGRAMMING

techniques used in linear algebra. In出ischapter， only a limiled， but useful discussion isp問 sented.First， it is important to understand LP problems，出巴modelingissues， and

their solution， as白eyare different from most of出eother problems in this book.

S配 ond，m佃 Yof由巳 currentnumerical techniques for NLP problems obtain their solution by linearizing the solution at the current design point. These linearized 叫uations回 nbe solved by白emeth吋 Sof this chapter.百le陀aderis dir田 tedto却 y

of the books on LP for a more detailed description of the models and the techniques as the p回sentationin山isbook is simple and brief. References 1-3 are useful for this

pu中ose.


Example 1.3 will be used to define the terminology associated wi白LPproblem.百le

modified versioD of the problem is used 10 develop出eformat of the LP problem so 由atwe can follow through with曲egraphical solution. The mathematical model has

two design variables xJ・X2thal represent白enumber of Component Placement Machines of type A and B. respectively. The objective is 10 maximize the number of boards to be manufactured. Constraint gl repr民間ts白eacquisition doJ]ars available.

Constraint g2 represen15白efloor space constraint. Constraint g3 represen15白enumbcr of operators availablc.

Maximize j(x): 990 XI + 900 X2 + 5250 (3.1)

Subject to: gl(x): 0.4 XI + 0.6 X2 S 8.5 (3.2)

g2(X): 3 XJ -X2 S 25 (3.3)

g)(x): 3 XJ + 6 X2 S 70 (3.4)

xJ ~O; X2~0

百leproblem. defined in Eq回tions(3.1)ー(3.4)is a natural representation of山

mathematica1 model in engineering design or血 yother discipline. By allowing 叫uality(=) constJ泊015組 dgr田 ter山anor eq凶 1(注)constrain15 the problem could

accommodate any mathematical model白atis characterized by linear functions. This representalion can be considered the inequality form of the mathematical model although it is not exp陀 ssedin由estandardformat for the LP problem.

3.1.1 S旬ndardFormat

The standard format of the LP problem includes only equality constrain15. It is sct up

as a minimization problem (somc authors prcfer a maximization problem). ln

addition. all of the variables in也emodel町eexpected to be semipositive (注 0)or nonnegative [4]. Finally， the cons位置intlimi15，出equantity on the right-hand side of

t

}露

3.1 PR08LEM DEFINITION 95

出econstraint， are required to be positive (> 0). In山eproblem defined in Equations (3.1)-(3.4)， the changes出atmust be carried out include converting the objective

function to脱 oppositetype， and transforming the constrain15 to恥 equalitytype.

Thev紅iablesXJ and X2， representing the number of Component Placement Machines

of model A and B， respectively， can be expected to be semipositive P-0)，白紙is，either

some machines will be ordered or no machines will be ordered. Note白紙the disαete

mωre of the problem is being ignor吋， as3.75 machines ofType A is not something

that would be ordered. The solution will be rounded to a suitable integer.τ1tere are extensive discussions in disc問胞 andinteger programming litera側隠 tosuggest白紙

白ismay nol be白eoptimum value. The justification for rounding off is白紙 itis

convenient 10 do so at this juncture.

In the objective function j(x)，出巴 coefficien15990 and 900 are called the cost

coe.伍cien15.Each coefficient白紙 isassociated wi白 adesign variable陀:presen15自己

increase (dec胞ase)in cost per unit change in the related variable.百lesimplest way to

紅白lsform山 o切ectivefunction to山erequired format is to multiply all of the terms

by -1. Therefore，出enew objective function is

Minimize j(x): -990 XJ -900 X2 -5250 (3.5)

In Equation (3.5)，山 constantterm on the right (-5250) can be absoぬedinto the

lefl-hand side by defining a new objective functionL(x) without affecting曲eopt加alva1ues for the design variables. lt can be observed出atincreasing the values of the

variables will make the objective more negative， which is good as we are trying to makej(x) as low as possible. If a particular cost coefficient had a positive sign， then increasing the amount of the corresponding variable would lead to an increase of the

objective function， which is nol desirable if白efunction needs to be minimized. This

idea is exploited in出eSimplex method presented later.

The泊。qualityconstraints have to be transformed to an叫ualityco田町aint.The

simplest wayωachieve this chang巴istoin回 ducean additional semipositive variable

for each inequa1ity constraint. Thal will create出E∞町田pondingequality cons甘剖nt

in a straightforward way. Consider出efirst cons凶 nt:

gJ(x): 0.4 xJ + 0.6 ~ S 8.5 (3.2)

τ五iscan be transformed to

gJ(x): 0.4 xJ + 0.6 ~ + x3 = 8.5 (3.6)

The new variable X3 is referred to as th巴 slackvariable. The definition is quile

appropriate because it takes up the slack of the original cons住aintterms when it is less

由加 8.5.τ1teslack wi1l be positive as出efrrst two terms will be less也佃or珂ualto

8.5 (for constraint satisfaction) from山eoriginal definition in Equation (3.2). In other

words， it is the difference betwecn山 constraintlimit (8.5) and恥 firsttwo terms of

the constraint. If tbe values of XI and X2 cause the constraint to be at i15 limit value.

出enX3 will be zero. This is the reason it is detined to be semipositive. By definition.

96 UNEAR PROGRAMMING

therefore， X3 is similar to the originaJ variables in the LP problem. Transforming the remaining constraints in a similar manner， the complete equations in standard format are

Minimize j(x): -990 x， -900 X2 -5250 (3.5)

Subj回 tto: 8'(X): 0.4 x， + 0.6 X2 + XJ = 8.5 (3.6)

82(X): 3 X， -X2 +X4 = 25 (3.7)

83(X): 3 x， + 6 X2 + Xs = 70 (3.8)

x， ~ 0; ~ ~ 0; x3 ~ 0; x4 ~ 0; Xs ~ 0 (3.9)

Equations (3.5)ー(3.9)exprl蹴 theLP problem in standard format. An inconsistency can be observed wi由 resp田tto曲eformat of the problems previously defined in the book. Namely， 8(") was reservedωdefine inequaJity cons位置intswhile h(，，) defined 珂uaJity∞m釘aints.Here， 8(X) is aJso being used for equaJity∞ns住aints.This change is acconunodated in白ischapter only. In subsequent chapters the prior meaning for the symbols is restored. Actually， symbols are not required 創出econstraint under disc凶 sion伺 na1so be児島rred10 by equation number. The formu1ation in Equations (3.5)-(3.9) can be exp問ssedsuccinct1y by resorting to a matrix notation.ηlis conversion is quite straightforward. Before embarking on山istransformation， the representation of the objective function needs to be chang'巴dagain. The constant term is brought to the righl-hand side so白紙由巳 obj田 tivefunction伺 nbe express叫 wi白terms only involving出edesign variables on the right. This is shown asL(x) below. Once again for convenience the same symbolj(x) is used in place of this new function in subsequent discussions. These extra symbols have been thrown in to keep the derivation distinct. This is a small price for the ability 10 develop血ernathematicaJ model in a natuml way and 陀constructit to conform to the standard rep陀sentationin most references. Therefore:

Minimize L(x):j(x) + 5250 = -990 x， -900 ~ (3.5)

Th巴followingdefinitions arc used to rewrit巴tIleproblem using malrices. Let the cost coefficients be repr，ωented by c，・C2，• . • ， c". The coefficients in出econstraints are represented by山esym加 1aij (using double subscripts).百lefirst subscript identi自民自巴 constraintnumber whi1e the second constraint is the same as the variable it is multiplying. It is aJso usefu1 to identify出巴日rstsubscript with the row number and the second subscript wi白thecolumn number to∞rrespond wi由 opemtions伽 tuse Iinear algebm. For example， the∞田町aint8，(x) wil1 be rep陀sented邸

g，(x): a" x， + a'2 x2 + a'3 x3 = 8.5

3.1 PR08LEM DEANmON 97

where all = 0.4. a12 = 0.6. al3 = 1. Because there are five design variables in山eproblem definition， additional terms wi白血巴 followingcoefficients， a'4 (= 0) and al5(= 0). are included in山eexpression for g，(x) without changing its meaning:

g，(x): all X， + a'2 x2 + a'3 x3 + a'4 x4 + a，sxs = 8.5

百leconstraint g2(X) can be simi1arly exp隠 ssedas

g2(X): Gz， x， + auろ+a23 x3 + a24 x4 +匂sXs= 25

The coe田cientsin the above equation can be回 silyestablished by comparison with Equation (3.7). Letting白eright-hand side vaJues of the constraints be represented by b" b2，・・・， bm• 出est血血rdformat in Equations (3.5)ー(3.9)c佃 be位叩slatedto

Minimize j(x): cTx (3.10)

(3.11) Subj即日0: g(x): A x=b

Side∞n柑 aints:"~ 0 (3.12)

x represents the column vector of design variables， including山eslack variables， lx" X2・・...x，，]Tσm戸田entsthe佐anspositionsymbol). c陀 presentsthe column vector of cost coefficients. [c" C2.・.• ，C，JT. b represents曲ecolumn vector of the constraint Iimits. [b" b2• • • • ， bm]T. A represents the m x n matrix of constraint coefficients. In 山isbook all vectors眠 columnvectors unless otIlerwise identified.

川MIN

--X

1川lJ什

lllつ

cc.c

r

・-EE

・E

・-E

・E

・-EEEa--』

一-pt，

-BElli--J

1

2

m

ムU

ムU

.

L

U

r'EBEEE'EEEEEEaEEE

・』

一-hu .，

、，EEEEEEEE'EaEEJリ」

M

h副

剛

a

a

a

ι1h・-W

1

2

m

a

a

a

j

rEEEEEEEEEEEEEEEEL

一一A

Comparing Equations (3.10) and (3.11) with Equations (3.5)ー(3.8)由evectors and matrix are

c = [-990 -900 0 0 O]T

b = [8.5 25 70]T

Azh?i :!| The foUowing additionaJ definitions have been used in the standard format. n is出E

number of design variables佃 dmis白enumber of constraints. In tlte example， n = 5 and m = 3. Nole tIlat the slack variables are included in the count for n.

98 LINEAA PAOGRAMMING

Negatlve Values of Design Varlables: The LP standard mathematica1 model allows only for nonnegative design variables. This does not limit白eapplication ofLP in any way. There is a standard procedure to handle negative values for variables. If a design variable， say x.. is cxpected to be unreslrIcted in sign，出enit is defined as出edifferenc巴oftwo nonnegative variablcs

11 ..1 x. =x-;-x I-^J J (3.13)

where x~ and x~ are acceptable LP variables. If the latter is higher than the former， xJ can have negative values. Equation (3.13) is used to replace x， in the math抑制calmodel. Note that白isprocedurc also increases thc number of design variablωin the model.

Type of Constraints: The less白anor equal to (::::) constraint was handled naturally in developing the LP standard mathematical model.百lestandard numerical technique for obtaining the solution requires that出eslack variables be posi“vetostart and b also be positive throughout. If there is a constraint白紙 isrequired to be maintained above a specified value， for example

g，(x): a" x， + aJ2 xz + a'3 x3 ~ b， (3.14)

then it appe紅snatural to introduce a negative slack variable -X4 to change Equ姐on(3.14) to an equality constraint:

g，(x): all X， + al2 x2 + a'3 x3 -x4主bJ

While it is possible to ch皿 ge血esign on X4 by multiplying with -1白roughout，白iscauses白eright-hand constraint limit to be negative， which should be avoided. In such cases， another variable， xs， is included in the exp胞ssion.In this case， two additional variables are now part of the model:

g，(x): a" x， + a'2 x2 + al3 x3 -x4 + Xs = bJ

τ'he variable Xs is called剖 artificialvariable. Artificial variables創芭 alsointroduced for回 chequality constraint白紙 isnaturally present in the model. The use of these variables wil1 be illustrated later through an巴xample.

3.1.2 Modellng Issues

The LP program is characterized by白emathematical model defin巴din Equations (3.10)ー(3.12).Due to the linear nature of a11 of the functions inv~lved， the discussion of the solution need only consider恥 coefficientsof the functions-the variabl邸

3.1 PAOBLEM OEFINJTION 99

themselves can be ignored. These coefficients， which are present as vectors or a ma回x，can be manipulated to yield solutions using concepts from linear algebra.

A look at the graphical solution to the LP problem suggests that the solution to the problem， if it exists， must lie on the constraint boundary because the curvature of the o吋ectivefunction is zero everywhere (it is linear). Furthermore， the solution to恥 LPproblem is primarily influenced by Equation (3.11). Being a Iinear equation. topics from linear algebra are necessary to underst加 dsome of出eimplications of白emathematical model.

Ifn=m，白atis， the number of unknowns is the same as the number of variables. there can be no scope for optimization as Equation (3.11) will determine恥 solution.provided one exists'. In the fol1owing. many of白巴 conceptsare cxplained through simple graphical illustration， or by consideration of simple examples. They mostly deal with examples of two variables (x" X2). For example， consid巴rtwocons釘aints

gl: xl+Xz=2

g2: -x1 +Xz = 1

Figure 3.1 illustrates the graphical solution to the problem. The solution is at x， = 0.5， and X2 = 1.5. This is independent of the objective function (whatever it may be). ln matrix form. A and b are

斗U

r--E

・E・-L一-

LU

--lE」

-EAt-‘

--A--且

『|「一一A

百leexistenc巴ofa solution depends on the rows of A (or columns of A-we will deal with rows only forconvenience). Ifthe rows of A are linearly independent， then there is a unique solution to白esystem of equations.

ThDefineanother problem as follows (only changing b 食omthe above example)・e new problem is

g，: x， +ろ=2

g2: 2xl+2x2=4

The new function g2 represents twice出evalue of gl・Thismeans g2 can be obtained 加 m8J' Therefore， 82 is said to depend on 81・Technically，出etwo巴quations釘elinear，かdependent一白eac加叫 dependenceof the functions is not a concem as much as the fact白創出eyar芭notlinearly independent. Both 81 and 82 establish the same line graphically. What is the solution to the problem? There are inflnite solutions， as long as the pair of (Xl・X2)values satisfies 81・InFigure 3.1， any point on the line representing 81 is a solution. Not only (0.5， 1.5)， but (2，0)， (0，2) are also solutions. From卸 optimizationperspective，白isis quite good for unlike the previous case of u瓜quesolution， the objective function can be used to find the best solution.1t would be better， however， if the choices were finite rather than infinite.

100 LINEAR PROGRAMMING

4

nH RM

“u

HM

内

u-ee

auy

・0

・lo

uu RM可

RH

U

内

42 m

内，

-ig

.

f

-

n

e

箇FM

，

3

21ー

)('" 1

。

ー1

-2

-3 J o

X1

2 -2

Flgure 3.1 Unique solution-Unear independence.

While severaJ cxamples can be constructed to illustrate linear dcpendence， it appears reasonable to assume thatfora凶 φ11回 ercise仇 optimization，the number of constraints cannot equal the number of ulIknowns (m手n).In the previous illus岡山n，n=2釦 dm = 1 (there was only one effective cons位aint).It is useful to recognize that 出iscorresponds to the case n > m.

If m > n， there are more equations than the number of variables.百lisimplies白紙

the system of equations represented by Equation (3.11) is叩 inconsistentset or has a redundant set of cquations. Consider the following illustration for m = 3 and n = 2 which uses the same 8¥ and g2 as in thc first illustration while adding a new 83:

g¥: X.+X2=2

g2: -x1 +X2 = 1

g3: X¥ + 2 x2 = 1

Figurc 3.2 illustrates白紙theset of equations is inconsistent since a solution doωnot exist. If one were to exist， then the three lines must pass through出esolution. Since 出eyar芭 alls釘泊ghtlines，出erecan be only one unique intcrsecting point In Figurc 3.2 only two of th巴threelines inter百ectat different points.

3

3.1 PROBLEM DEFIN町10N 101

ペメ

FV一予一一一vb//之

K

A

A

昨年

md引引訓ゴ////

4

Case n a 2; m a 3: No solutlon; In∞nslsten! 5e!

3

)(~・ 1

。

ー1

-3 4 -2 -1 o

X1

2 3

Flgur由3.2 No solulion， Inconslstent set 01 equations.

Rcdefine g3 as

g3:XI +2ろ=3.5

This new 83 is山edashed line in Figure 3.3. Now a uniqu巴solutionto出eproblemat (0.5， 1.5) is established.百lisis山 othe solution established by consideration of 8¥ 叩 dg2 alone. This implies白紙 g3is redundant. If 81 is multiplicd by 1.5， 82 is multiplied by 0.5 and both added. thc result is g3 defined above. That is，

g3 = 1.5 gl + 0.5 g2

ぉcanbe obtained by linearly combining g¥ and g2.白紙 is.adding constant multiples of仰向nctions.This is another example of linear dependence. This linear dependence and redundallCY is also iIIustrated by the set

g¥: X¥ + "2 = 2

g2: -xけら=1

g3: 2x1 .. ・2x2= 4 (same as gl)

102 LlNEAR PROGRAMMING 3.1 PROBLEM DEFINITION 103

For n == m， and with gl: xl + x2 == 2， and g2:ーX1+X2 = 1，出ematrix A is

Case n a 2; m. = 3: UnlqU8 solullon， Unoar dependonc9 -EEEEE『EJ

1

1

「|卜』=

A

4

h

h 、-

h -

E-「角

a and its detenninant is

det(A) = IAI = (1)*(1)一(1)*(ー1)=2

-・ "3.5

From theorems in Iinear algebra [5]， if det(A) is not zero， in which case the matrix A

termed nonsingular， a unique solution exists to the set of equations. This was町ucfor 白eexample iIIus回 tedabove， as恥 onlysolution w俗 locatedat (0.5， 1.5)， shown graphically in Figure 3.1.

Consider the example with gl: XI + X2 = 2，叩dg2: 2x1 + 2x2 =4. In山iscase，

.r 1

。

det(A) = IAI = (1)*(2)ー(1)ペ2)== 0

-3 -3 -2 O

X1

2 3

lf det(A) is zero， that is， matrix A is singular，恥 reare either no solUlions or infinite sollllions. For山isexample，出ere were infinite solutions. Dctcrminants of higher心rdersqu町'ematrices are evaluated by setting up lower-order determinants

until出eyare陀ducedto a 2 x 2 determinant， which is evalua民das above. Any

textbook on engineering mathematics or linear a1gebra should illustrate白istechnique. It is not回producedhere.

If n "" m， discussion of the existencc of solutions requi隠 sadditional concepts Iike rank of a matrix and augmenled matrix. Since由巴 casen > m is of interest in

optimization， only that case is employed in subs叫uentilIustration. A useful

discussion will need at 1east three variables and two equations. Three variables will deny the use of graphics to develop the folIowing concepts. Using the set

Flgure 3.3 Unlque solutlon， IInear dependence.

This suggests出創出econcept of redundancy can be associated with linear

dependence. The discussion of linear dependence and independence was established

凶泊gequations above. The same discussion can take placc by reasoning on the coefficicnts themselvcs. In this case the coefficient ma凶xA should lead us to血esame concIusion rcgarding山elinear independence of a set of Iinear equations.百 econcept of a delerminanl is n田 cssaryto develop the criteria for linear dependence.

gt: x1 +X2 +X3 = 3

g2: -Xけら+0.5 x3 = 1.5

Determlnant: The determinant is associated with a squarc matrix. For a general 2 x

2 matrix A， where the matrices紅白

可EEEEBEE-J

勾

，

働

帽

4

・E内，.

aa

-E・内，a

rMlML

=

A

可EEE

・E

・-，d

3

5

-『M

1

・nu

I

l

--

「|仁一-

-A

可EEEEEEJ

3

5

--a

FEE-EEaE』

Z

hu

--EE

・E.

，JIω

I

l

--a--a

flトL

=

A

(3.16)

|a a 1いいいい=吋吋叫吋d白似削巴d叫t(仏ω山A削ル山門)=同=吋aa句21 句2

The new matrix A. is called出eaugmented matrix-the columns of b are added to A.

According 10山∞remsof linear algebra (presented here without proof): 出edeterminant is expres'ied and evaluated as

(3.15)

• If the augmented matrix (A・)and the matrix of coefficients (A) have the same rank r<n，山en出er巴M巴manysolutions.


. If曲eaugmented ma凶x(A・)加d出ematrix of coefficients (A) do not have the same rank， a solution does not exist.

・Ifthe augmented matrix (A・)and the matrix of coefficients (A) have the same rank r = 11， where m = n， then there is a uniqu巴solution.

Rank 01 a Matrix: While there are formal definitions for defining the rank of出ema凶x，a useful way to determine出巴rankisωlook鉱山edeten凶n剖t.τ'herankofamatrix A is the order of出elargest nonsin肝larsquare submatrix of A， that is，出elargest submatrix with a determinant other白anzero.

In the example above， the largest square submatrix is a 2 x 2 matrix (since m = 2 and m < n). Taking the subma住ixwhich includes the first two columns of A，出edeterminant was previous established to have a value of 2， therefore nonsingular. Thus， the rank of A is 2 (r = 2).百lesame columns appear in A市 makingits rank also 2.官lerefore，infinitely m血 ysolutions exist (r < n). One way ωde民rmine白E

solutions is to assign (n -r) variables arb加的 valuesand use them to determine values for the remaining r variables. The value n -r is also often id巴ntifiedas出巴

deg ree of freedom (DOF) for the system of equations. ln the particular example above，山eDOF can be identified with a value of 1 (i.e.， 3 -2). For instance， x3 can be assign巴da value of 1 in which case XI = 0.5叩 dX2 = 1.5. On th巴otherhancJ. if X3 = 2， then XI = 0.25 and X2 = 0.75. The above cases illustrate that for a DOF of 1， once x3 is selected. Xl阻 dX2 can be obtained through some further processing.

Another example for iI1ustrating曲erank is出einconsistent system of equations in釘oducede紅 lierwhere n = 2叩 dm = 3， reproduced for convenienceω

gl: x1 +x2=2

g2: -x1 +ろ=1

g3: x1 + 2x2 = 1

The matrices for this system are

A=H l} b=[I} A'=H i l] The rank of A cannot be greater白血 2，since n = 2 and is less th加 m= 3. From prior calculations出edeterminant of the first two rows of A is 2. This is gr凶 terthan zero and th巴reforethe rank of A is 2. For A. the determinant of the 3 x 3 matrix has to be examined. The determinant of a general n x n square matrix A Can be calculated in terms of the cザ'actorsof the matrix. The cofactor involves血ede飽nninantof a minor matrix-which is an (n -1) x (n -1) matrix ob凶nedby deleting an appropriate single row and a single column from the original n X n ma住ixA， multiplied by 1 or -1， depending on曲eto凶 ofthe row and column values. References 5 and 6 c胡 providemor巴information.Here，出巴determinantfor the 3 X 3 matrix is defined for iIlustration

3.1 PROBLEM DEFINITION 105

in出ebook.百le(+1)加d(ー1)below ar芭 obtainedby using (ー1);ザ， wher巴iandj represent出erow and column values of the coefficient multiplying the value

al1 a門 al"ll ー -

A= 10;; lln iln卜(+l)a111273|+(ーl)a1212J2|+(+l)a13仕ご|~1 ~2 a

331 品目 I~. ~， I 1--~ I -~:l l

(3.17)

百lerefore，

2A

司

L

--a--a

，.且+

・・且

-za

'Ea'EA

tA

-aa--A

1

1

'・a一-

-EA--A2且

'且'且今，M

I

l

l

一一-A

= (1)(0) + (-1)(-2) + (1)(ー3)=-1

A. is nonsingular and its rank is 3 whiIe the rank of A is 2. The system of equations will have no solutions from the above th巴orcmand can be seen graphically in Figure 3.2. For the sake of completeness， using the previously defined dependent set of equations:

gl: x1+x2=2

g2: -x1 +x2 = 1

g3: 2x1 + 2xa =4

which yields the following ma凶ces:

A=[-l l} b=[!} A'=H 1 !]

The rank of A is 2. Note白紙usingthe first and last row of A for the calculation of the determinant yields a singular 2 x 2 subma凶x.The determinant of A・is

nu --

00

ζ

U

+

今ゐ

=

'A今

4

2A司，B、.，

司品(

+

'且A『

寸

2、.

E

，，ート+

'ia品T

a且勾，“

、.z'Z且''z

、-一

内，

a'Aa且守

'1

・1司L

'・A--a--E・

-一-A

which makes A. singular. However， the rank of A・isat least 2 due to出巴presenceof曲目阻lesubma佐ixwhich determined the rank of A. From the above theorem，血er叩 kof A is equal to the rartk of A.， which is 2 and this is less出制m= 3. It can be concluded 伽 t出ereare infillite sol凶 onsfor山 systemof equations.


A useful observation of matrix A・回thep陀cedingexample is白紙twoof the rows

are a constant multiple of each other.百世smakes出ema回cessingular. The same

holds for two columns出atare multiples of each other.

Un/t Vectors: Real-world optimization problems frequently involve a large

number of design variables (n variables). In real LP problems.白enumber of variables

can be staggering. For a successful solution to出eop也凶zationproblem. it is

implicitly assumed that the set of design variables is linearly independent. That is.

there is no direct dependence among any two variables of the sct. In出iscase. the

abstract ~pace spawned by the design variables is cal1ed白en-dimeflsioflal Euclidean

space. The word abstract is uscd herc to signify that it is a construct in the imagination， as only three variables can be accommodated in由巳 familiar3D physical space.百le

design vector is a point in白isspace. The Euclidean spa田 isa direct extrapolation of

the physical， geome凶cal3D space伽 tsurrounds us. This space is termed恥

Cartesian space. For an i11ustration， consider the point (2，3，2) in Cartcsian space or in

the rectangular coordinate system. In Figure 3.4， the point is marked P. The凶adof

numbers within parentheses denotes the values along the x" X2・andX3畝 es，陀spectively.

The Cartesian space is identifi吋 bya set of mutually pe叩endicul釘 linesor axes

markedx"x2' andx3・Thepoint wherc they intersect is the origin (0) of the coordinate syslem. The point P can also be associated Witll the vector OP. drawn from tlle origin

o to血epoint P. Very often a point and a vector-two very different entities-are

used synonymously. From elementary vector addition principles， the vector OP can

becons回 ctedby consecutively joining three vectors along the thr田 coordinate出 es.

In the figure this is shown by laying out a vector along the x， axis， adding a vector

along出eX2 axis to it， and finally adding thc third vector along thc X3 axis. The additional small arrows in thefigure are白eLmit vectors.百leyhave a magnitude of

one and a direction along each of the蹴 es.These unit vectors are labeled e" e2， and e3 in the figure. Mathematically，出巴 pointP (2，3，2) and the vector addition can be

rcpresentedω(column vectors)

X3

2

3

X2

Flgure 3.4 Rectangular coordlnate system.

3.2 GRAPHICAL SOLUTlON 107

崎‘de

ヲ白+

q

，-e

43 +

dL'

ヲ“一一

可EE--EEEBEESEEJ

nunu'E

FEE--EE

・E

・-BEE-L

今ゐ+

可EBB-a'EBEE--J

nu'且

nu

rlll111L

43 +

1111111」

'znunu

r'EEEaEaE

・E

・-EL

内

L=

守EEEEEEEEEEEEJ

rhHhμ

いい

(3.18)

The advantage of unit vectors is白紙曲eycan cstablish加 ypoint in the Car飽sianspace

through an approp巾 tetriad of numbers sig凶fyingcoordinate values or 10回 tion.Acωally， any point in由eCartωianspace can be detennined by any set of thl国linearly independent vectors. These vectors are termed the basis vectors. In this case. the simple connection between the point and the elementary addition， illustrated above， will not be availabl巴.Fu巾 ermore，伽oughthe methods of Iinear algebra， these three vcctors can be reduccd to the unit vcctors through elementary row/column

opcrationsー alsocalled Gauss-Jordan elimination or reduction. In n・dimensional

Euclidean space spanned by the n design variables，白epoint p"(p.. P2， . . . ，p")， will be represented by a collection of n values enclosed in par官n白eses.The corresponding 目unitvectors are d巴finedas

o o

e"=I.

(3.19)

官levector from由巳origin(0) to P" is陀P問sentedas

OP = p，e， + P2e2 + . . . + p"ell

These new definitions will be sufficient to understand the techniques used to solve LP problems.

3.2 GRAPHICAL SOLunON

百legraphical discussion is again limited 10 two design variables in由巳 originalproblem description. It is used here to develop certain geometric ideas related 10 LP. InLP， th巴introductionof slack variables will easily exceed this limit of two variables for graphical ill凶住ation.so the graphical solution is illus回 tedwith民 spectto出e

original variables in the design problem. MATLAB is once again used to合aw出e

graphical solution. The problem is not reduced 10 the standard format for the graphical

solu“on. It is important to do 50 for a numerical solution. Since Ihe primary graphical element is出estraight li1Le， a function for drawing 1ines is developed through th巴function m-file drawLine.m below.百 isfunction is also used to provide more exposぽ巴 toMATLAB programming.

drawLlne.m:古山 isa function m・filefor drawing str泊ghtlines.百le1ine 10 be drawn is represented as

108 LlNEAA PAOGAAMMING

以 +by=c

Since x， y are usedωsymbols for variables， xl represents the lower range on x， while X2 represents the upper rangc. Limiting Iines/constraints can bc drawn by setting either a or b to zero. Color (red) is uscd to draw a p町'allelline at 10% increase/decreasc in the original value of c (depending on the direction of the original inequality) instead

of hash marks. Limit constraints are in magenta and th巴objectivefunction is in blue dashed line. The new MA 11.AB command here is出eline command. You are

enco町 agedto s田 khelp on the command. The code:

drawLine.m 者 Drawinglinear constraints for LP programming problems 者 Dr. P.Venkataraman 者 OptimizationUsing Matlab 告 Chapter3 ・ linearProgramming も

者 Linesare represented as: ax + by c ( c >= 0) 者 x1，x2 indicate the range of x for the line 者 typindicates type of line being drawn 1 (<=) も 9 (>=)

告 n (none) 者

苦 Thefunction will draw line(s) in the figure window 者 thegreen solid line represents the actual value % of the constraint 者 thered dashed line is 10者 largeror smaller 者 (inlieu of hash marks) 者 thelimit constraints are identified in magenta color 苦 theobjective function is in blue dashed lines 者

function drawLine(x1，x2，a，b，c，typ) 告 recognizethe types and set color if (typ 'n')

str1 'b'; str2 'b' cmult 1;

else str1 '" 'g' str2 '" 'r';

end

者 valuesfor drawing hash marks 者 dependingon the direction of inequality if (typ -= 'n')

if (typ == ' l' ) cmult +1;


else cmult -1; end

end

告 setup a factor for drawing the hash constraint if (abs(c) >= 10)

cfac 0.025; elseif (abs(c) > 5) & (abs(c) < 10)

cfac 0.05; else

cfac 0.1; end

if (c 0)

cdum cmult*O.l; else

cdロm (1 + cmult* cfac)*c; end

苦 ifb 0 then determine end points of line x line if (b -= 0) y1 (c -a*x1)/b; y1n (cdum -a* x1)/b; y2 (c -a* x2)/b; y2n (cdum・ a*x2)/b;

else

者 identifylimit constrainヒsby magenta color str1 'm' str2 'm';

y1 x1; 者 sety1 same length as input x1 y2 x2; も sety2 same length as input x2 x1 c/a; 苦 adjustx1 to actual value x2 c/a; 苦 adjustx2 to actual value y1n '" 0; 者 sety 0; y2n 0; 者 sety 0

end

if (a 0)

str1 'm' ;告 setcolor for limit line str2 'm'; も setcolor for limit line

end;

者 drawaxis with solid black color

hh line( [x1，x2]， [0，0]);


set(hh，'LineWidth' ，1，'Color' ，'k');

hv 1 ine ( [0 ， 0] ， [xl， x2] ) ;

set(hv，'Linewidth' ，1，'Color' ，'k');

者 startdrawing the lines

hl line ([xl x2]， [yl， y2] ) ;

if (t乎p 'n')

set(hl，'LineWidth'，2，'LineStyle' "ーー'， 'Color'，strl); else

set(hl，'LineWidth' ，1，'LineStyle'，'・'， 'Color'，strl); end

if (b -= O)&{a -= 0) text(x2，y2，num2str(c)) ;

end if ( b == 0) I (a == 0) I (ヒyp 'n')

grid return， end

grid; h2 line ([xl x2]， [yln， y2n] ) ; set(h2，'LineWidth' ，0.S，'LineStyle'，':' ，'Color'，str2); grid hold on

3.2.1 Example 3.1

The problem from page 94 is reproduced 00田 morefor cooveoieoce.

Maximize j{X): 990 XJ + 900 X2 + 5250 (3.1)

Subj回 tto: g.(X): 0.4x. + 0.6x2 S 8.5 (3.2)

g2(X): 3xJ -X2 S 25 (3.3)

めは):3x. +6x2S70 (3.4)

x. ~ 0; .¥i ~ 0

Th巴graphica1solutioo is shown in Figure 3.5. The solution， rcad from thc figu飽 (youC姐 Z∞m白earcaof solutioo)， isxj = 10， andxi = 7. It is出cintersection of the active constraints (3.3)叩d(3.4).百leactual values are xj = 10.48加 dxi = 6.42. Since an integra1 number of machines have to be ordered， the solution is adjustedωa

neighboring integer va1u巴由atsatisfies lhe contraints.

8Or'

60

3綿』

-40 0

3.2 GRAPHICAL SOLUTION 111

5

~~

.. 句ー

-，・--守 .. ・.ーー・・』ーー一 ..

句・.‘ ・.旬、

‘ .. ・... h ・... ー-一一一一一二三.J-52505 10 15 20 25 30

Numbor 01 machlnas 01 Iype A

Flgure 3.5 Graphlcal sOlulion， Example 3.1.

3.2.2 Characterlstlcs of the Solutlon

τbeglωmetry evid巴ntin the graphical solution of Example 3.1 is used to explain some of the concepts associated with LP and ilS numerica1 technique. Figure 3.6 is白E

graphicaJ rcp問senlationof曲巴 constraintsinvolved in Example 3.1 (same as Figure 3.5 without the objectivc 釦nction).In order to 隠 late出egcometryωLPconceplS， thc stan也rdforrnat of LP is n配 ωS釘y.The standard forrnat established before is

Minimize j{X): -99OxJ -900x2 -5250 (3.5)

Subject to: g.(X): 0.4x. + 0.6x2 + x) = 8.5 (3.6)

g2(X): 3x. -X2 + X4 = 25 (3.7)

g3(X): 3xI + 6x2 + Xs = 70 (3.8)

-x. ~O; x2~0; X3~0; X4~0; x5~0 (3.9)

In Figure 3.6 the constraints x. ~ 0 and X2 ~ 0 are addcd to the three functional constraint Iines (3.6)-(3.8). The ha.'!hed area is thc feasible region，出atis， th巴design

112 LlNEAR P伺OGRAMMING

50

40

30

! 20 右

i 10 2 百

-30 0 5 10 15

Numbor 01 Machln08 01 Typo A 20 25

Flgure 3.6 Feasible region， Example 3.1.

space in which all of the constraints are satisfied. The circles exaggerate the points of intersection of five constraints taken two at a time. Many of them are numbered Pl， P2， . . . ， P6. AII of these points of inter富田tioncan be associated with a certain property

conceming the values of the variables. Note there are five dcsign variables (n = 5) and threc functional constraints (m = 3). For e瓦創nplc:

PI: (x1 = 0， x2 = 0， X) = 8.5， x4 = 25， Xs = 70)

P2: (x1 = 0， ~ = 11.67， X) = 1.5， x4 = 36.67， Xs = 0)

P5: (xJ = 10.7， x2 = 7.05， X) = 0， x4 = 0， Xs = -4.4)

P6: (x1 = 0， x2 = 14.17， x3 = 0， x4 = 39.17， Xs =ー15.42)

The values of出evariables are obtained as由einte路氏tionofthe constraints taken two

at a time. In出eabove Iist， for each point， exactly two of the variables are zero. The number 2 corresponds to the value of n -m. n represents the number of variables and m the number of constraints. Points P5 and P6 are infeasible because one of the variables has a negative value.

Baslc Solutlon: A basic solution is one obtained by setting cxactly n -m variables to zero. In Figure 3.6， aIl of the points identified by the circles rcprcscnt basic solutions. The points choscn above are all basic variables. In general， for n design variables and m cons佐官ints，出巴numherof basic solulions is givcn by the combination

3.2 GRAPHICALSOLUTION 113

(三)=ポ可This would yield 10 points for our ex儲nple，which is shown in Figure 3.6.

Baslc Varlables: The set of variables in the basic solution that have nonzero values

are called basic variables. Correspondingly， the set of variables in the basic solution 白紙havethe value of zero are called nonbasic variables. For the point Pl， XJ血 dX2are nonbasic variables while X3・X4・andxsar泡 basicvariables.

Basic Feaslble Solutlon: This is a basic solution that is also feasible.τ'hese are

出epoints Pl， P2， P3， and P4 in Figurc 3.6. In LP， the solution to the problem， if it is unique， must be a basic feasible solution. The basic solution can aIso be considered

geome凶caIlyas a corner point or an extreme point of the feasible region.

Convex Polyhedron: This is a bounded region of the feasible d巴signspace-白e

rcgion defined by出巴quadrilateralcomprising the points PI， P2， P3， and P4 in Figure 3.6. The term cOllvex set represents a collection of points or vectors having the following property: For any lwo points in the sct (or within a rcgion)， if all ofthe points on the linc connecting血巴twopoints also Iie in出csame region， the region is a convex set. Imagin巴削yline drl1wn in lhc region tl巴finetlby山equadrilatend whose comer points are Pl， P2， P3， and P4. From Figure 3.6 it is c1ear the line wilI still be wi白in

the region established by臨 qua合iJater叫-making it a convex set.

Optlmum Solutlon: This is a feasible solution出atminimizes the objective

function-point P4ー inFigure 3.6. In LP， the optimum solution must be a basic

feasible solution.

Basls:τ'he basis陀 P隠 sents出ccolumns of the coefficient matrix A白紙coπespondto the basic variables. They form thc basis of the m-dimensionaI space. They are

termed the basis vectors.

//J6422plz語ガ/4124哨巾hr一YY

ー

Canonlcal Form:τ'he basis vector宮間duc吋 tounit vectors through row/column operations (or Gauss-Jordan eIimination). The basic feature of出enumerical technique for LP (Simplex method) is a隠 petitiveproced町 estarting from an initial

basic feasible solution， and determining出ebest neighboring basic feasible solution that improves the objective. The procedure is carried on until the optimum sol凶 onis

reach巴d，or if it is determined白紙nosolution is possible. Thc canonical form is used

for rapid1y identifying the solution. Each iteration in出巴 procedurecan be described

mathematicaIly as follows.

The starting constraints創'corganized as

[A]II附[丸田1= [b]附

After the Gauss-Jordan elimination， Eq~Jation (3.20) is assembled as

(3.20)

114 LlNEAA PAOGAAMMING

凹m糊[X]m+[R]州 n-m)[X](s弔問)=[b]

The set of m design variables in the first term副官 thebasic variabJcs. The sct of (n -

m) dcsign variabJes in出csccond tcrm are肱 nonbasicvalucs. In出cSimpJex method， thc nonbasic variablcs are summarily set to zcro.

3.2.3 Di偽陪ntSol凶 onTyp関

Thcrea回 atmost four different resuJts白atcan be expccted for the soJution of the LP

probJem: (1) a unique solution， (2) infiniteJy many soJutions， (3) unbounded solution， and (4) the possibility山atthere is no soJution.

(1) Unique solution: 百leexample used for discussion has a Usique solution

(Fi割問 3.5).百lecondition necess紅 yfor血isto occur is山鉱山eobjective

function and the constraints have dissimilar sJopes， and the feasibJe region is boundcdlcJosed. Geometrically， this can be visually expJained as the movement of山eline represcnting thc objective function， pぽullcJ10 itseJf， in a favorabJe

direction， until it remains just in contact with one of the feasibJe comers (bωic feasibJe soJution) of the feasible region. The fe悩 ibJeregion is identified uistinctly in Fig町宮 3.6.A simpl巴巴瓦巴rciscuf imagining diffe印刷 objectiv巳

function Jines should convince the viewer that several different objectives伺 n

be defined to locate the unique solution at P 1， P2， P3， or P4. This 20 geome町

and construction can be used to understand the extension to n dimensions. He児the Iines wilJ be represented by lIyperplanes.

(2) lnfinile solution:・ Inorder for出isto occur，出eobjective function must be

parallel to one of the cons位置ints.For example， in Figure 3.6， let the original problem be redefined so白紙theobjective function is parallel to the constraint

h・Anypoint on the constraint， and lying between the line segment dcfined by 出epoints p2 and P4， is an optimal solution to the problem. They will yieJd出e

sarne value of白州問ve.

(3) Unbounded sollllion: In出iscase， the feasible region is not bounded. In

Figure 3.6， if the constraints gl and g2 were not part of the probJem formuJation，出enthe feasibJe region is not bound on the top. Referring to Figurc 3.5.出E

obj民 tivefunction can be shiftedωunlilTUted higher values. In practice， there will be an upper bound on出巴 rangeof the design variabJes (not p訂tofstand紅d

format) that wilJ be used to cJose山isregion. in which case solutions oftype (1)

or (2) can be recovered. The prcsence of an unbounded solution a1so suggcsts 伽 tthe formulation of批 problemmay be lacking. Additional meaningful

constraint(s) c叩 beaccommodated to define the soJution.

(4) No solulion:・Figure3.6 is used to explain血ispossibiIity. Consider白紙白E

direction of inequality in 81 is changed to thc opposite type (注).The feasiblc

region wi出rcspectto the constmints gl and 83 is to the right of XI = J 5.0. This

3.3 NUMERICAL SOLUTION-THE SIMPLEX METHOD 115

point is not feasible with respect to出econstraint 82・Therefore，there is no point 白紙isfeasible. There is no solution to山eproblem.

In the above discussion.出etwo-variable situations provided an obvious

cJassification of the solutions. In practice. LP models are large with over hundreds of

variabJcs. Modeling and transcription errors may give rise to many of the above

situations. The genemtion of solution is based on numerical tcchniqucs of linear algebra， which is often sensitive to山equality of the matrix of coefficients. FiJtering out errant data is usually a significant exercise in the search of optimal solutions.

3.3 NUMERICAL SOLUTION-THE SIMPLEX METHOD

The standard numerical procedure is based on山ealgorithm due to Oantzig笛

mentioned earlier. It is refe町叫 10as出eSimplex method. The procedure is related to白巴 solutionof a system of Iinear equations. The actual application of白巴

procedure can be associated with the Gauss-Jordan method from linear algebra.

where the coeffici巴ntrows are transformed through e1ementary multiplication and addition. Most mainframe computcr installations usually carry software that will help solve LP problems. MATLAs also provides procedures 10 solve LP problems in its Optimization TooJbox. In this scction， the Simpl巴xmethod is applied to simple probl巳m昌primarilytu undcrstunu thc pl'Ogrumming and geomctric features. This will be used when we discuss direcI techniques for nonlinear problems. In the next section， the Simplex method is introduced in detaiJ with explanations. In subsequent sections， MATLAB. or spreadsheet programs Iike Excel can also be used to implement the SimpJex method.

3.3.1 Features of the Slmplex Method

In this section. the machine seJection ex創npJeintroduced earJier is set up for applying the Simplex mc白od.It is instructive to notc the method is iterative. Given a starting point， it will march forward through improving designs un臥 ithas found the soh瓜onor cannot proceed further. For completeness the original problem is rewritten here:

Maximizc j(X): 990xl + 90Ox2 + 5250 (3.1)

Subject to: 81(X): O.4xl + 0.6X2 :s; 8.5 (3.2)

(3.3)

(3.4)

45Fir-2」TIz--ι

「

g2(X): 3xI -X2 :s; 25

83(X): 3xI + 6x2:S; 70

x1 ~O; x2~0

The problem was aJso transformed 10 the standard format as follows.


Example3.1

Minimize J(X): -99OxI -90Ox2 -5250

Subject to: g，(X): 0.4xl + 0.6x2 + x3 = 8.5

(3.5)

(3.6)

g2(X): 3x， -x2 +X4 = 25 (3.7)

g3(X): 3xI + 6x2 + Xs = 70 (3.8)

x1き0; x2 ~O; x3注0;x4さ0; Xs注o (3.9)

X3， X4， and Xs ar芭曲eslack variables.

The Simplex method is used on the problem being expressed in the standard format.Thefollowing information is useall in organizing the calculation as well as

陀cognizingthe motivation for subsequent iterations.百1eserefcr to a problem for

which a unique solution exists. While m阻 yof the items bclow wcrc in加 ducedearlier， they are referenced here for completeness.

• Thc number of variables in thc problem is n.羽山includesthe slack and surplus variables.

・百1enumbcr of constraints is m(m < n).

・百1eproblem is a1ways to minimize the objectivc functionf

• The number of basic variables is m (same as出巴 numberof constraints).

・Thenumber of nonbasic variablcs is n -m.

・ The set of unit vectors expected is e" e2，・・・， em•

• The column of the right-hand side (b) is positive and greater白anor equal to zero.

・Theca1culations are organized in a table， which is called a tableau， or Simplcx tableau. It will be referred to鎚 thetable in this book.

・Onlythe va1ues of thc c佃仔icientsar官necess町yfor the calculations. The table 中creforecontains only coefficicnt va1ues， the matrix [A] in previous discussions. These紅巳 thecoefficients in the constraint equations.

・百1eobjective function is出elastrowin出etable.百1econs住温intcoefficients are written自民t.

• Row operations consist of ndding (subtracting) a definite multiple of the pivot row to othEr rows ofthetable.The pivotrow identifies the row in which theunit vcctor (in a.ccrtain column) will havc the value of 1.

• Row operations are needcd to obtain the canonical fonn. Practically this回plics白紙 youshould bEable to spot thEunit vectors in the table.If this happens，

then the current iteration is compl巴teand considerョtionsfor the next onc must begin.

3.3 NUMERICAL SOLUTlON-THE SIMPLEX METHOD 117

With this in mind， the Simplex method is applied to the machine selection problem. Thc example will use and cJarify the items in the above Iist.

3.3.2 Appllcatlon of Slmplex Method

Simplex Table 3.1: In Table 3.1 (Simplex Tnble 1)，出cfirst row indicatcs the variable names. The last row is出eobjective function. Spreadsheets are an efficient

way to pro田 ssthe infonnation.百 itis being cmployed，白en由巳 symbolf can be removed from出elast column and last row to allow numerical ca1culations (ぉshown).ThelnstcoluD1nis出巴right品nndsid巴values.The rest of thc cntries紅白thecoefficients ofthe cons住aintequations.

The current itcration is over if the table displays the canonical fonn. In practice白ecanonical fonn comprisωspotting the m unit vcctors in the table， as well箇 making

sure the en凶esunder the b column， except for the rows representing白eobjective

function(s)， are nonnegative P-Q). A glance at Simplex Table 3.1 indicates伽 t出E

canonical fonn is present. Thc unit vectors in出ctable also idcntify those m variablcs出atwill belong to the

b錨 is.Those variables will have a nonzero value for出isiteration. The remaining， n-m， nonbasic variables are sct to zero. This solution is directly intcrpreted合omTable3.1. The solution therefore is

x， = 0.0， xz = 0.0，ろ=8.5， x4 = 25， Xs = 70，叩df=-5250

Has the Optimal Solution Been Ob飽ined? The optimnl solution requires that f be reducedぉ muchas possible. For the current solution白巴 valueof XI is O. From Equation (3.5)， if x， were increased from zero，出enthe objective functionf decreases

further in value. Similar reasoning cnn be used for X2・Therefore，the solution can be further in司proved.Solution hns not converged. The second/next iteration will問quirenssembling another table， Table 3.2.

The only way for x， and/or X2 to have a positive value is if自己yb田 amea basic variable. Since the number of bnsic variables is prescribed for the problem， some of 出ecurrent bnsic variables must become nonbasic. For each iteration，出eSimplex method a1lows only one pair of variablcs to conduct出isbasis-nonbasis exchange. Each iteration takes place through a new table. To affect the exchang巴itis necessary

Table 3.1 Simplex Table 3.1

XI X2 XJ X4 X~ b

0.4 0.6 。。 8.5 3 -1 。。 25 3 6 。。 70

司 990 -9∞ 。。。 5280

119

Simplex Table 3.2 (Table 3.2): The pivot row is obtained from origin叫 row20f

Table 3.1. The following is白巴originalrow

25

The value of the first element in the new row wi1l be 1. The row is modified by

dividing through by 3. The new second row in Table 3.2 will be

8.3333

In Table 3.2，血efirst element of the first row (Table 3.1) must be reduced to O. This

is always achieved by adding/subtractingfrom the row being modified an appropriate multiple ofthe pivot row. The modification to白efirst row is

row 1 -[0.4 " (pi vot row 2)} to yield

o 0.73331 -0.1333 0 5.1667

Thefour出rowwi1l be obtained by

row 4 + [(990 " (pivot row 2) I

Table 3.2 is obtained after these calculations. The negative value of the coe:悶cientin Table 3.2 in row 4 indicates白紙weneed

at least one more iterationltable. The EBV is X2・Byexamining白巴positiveratios of 曲eva1ues in白巴bcolumn to the coefficient in白巴X2column， the LBV is id回 tifiedas

Xs・Thepivot row is row 3. Table 3.3 wi1l be constructed by transforming the rows of

Table3.2.

Simplex Table 3.3 (Table 3.3)

pivot row 3 = (row 3)刀new row 1 =row 1-{0.73333 * (pivotrow 3)} new row 2 = row 2 + {0.3333 * (pivot row 3)} new row 4 =row 4 + {1230場(pivotrow 3)}

Table 3.3 is obtained based on the above operations.

Simplex Table 3.3 Table3.3

b

nu---nunu


。。3

O 0.3333 。一0.3333

The third row will be obtained by

row 3 -{3 "(pivot row 3)}

0.4524 10.4762 6.4285

21437.14

Xs

-0.¥047 0.0476 0.1428

175.71

X4

-0.0285 0.2857

-0.1428 154.28

X)

'且

nunU凸

U

X2

nunu--AU

XI

LlNEAR PROGRAMMING

to determine which is the variable白紙 willenter the basis (XI or X2)， Entering Basic Variable (EBV). Also to be established is which of白evariabl回 (X3'X4' or xs) will

leave恥 basis，Leaving Basic Variable (LB V). In terms of the numerical calculations， in the new table， the unit vector under the LBV column wilI be transferred to the EBV

column through elementary row/column operations.

118

Entering 8asic Variable: The EBV is chosen by its ability to provide for the largest

decrl団関inth巴objectivefunction in也ecurrent iterationltable.百lisis detercnined by

the largest negati ve coeぽicientin the row associated with the objective function. From

Tabl巴3.1，this value is -990 and co町邸pondsto由evariablc XI・EBVis X!. A tie can

be brokcn arbitrariJy.

Leaving 8asic Variable: The LBV is also chosen by its abilityω 泊lprove血e

o町田tive.The LBV is only determined after the EBV has been chosen.官邸 is

important because only the column under出eEBVisexaI凶nedto estabIish血巴LBV.

The minimum l'atio of the values of出eright-hand side (b column values) to the

corrcsponding coefficients in the EBV column， provided出ccoefficicnt valuc is

positivc， d，巴cidcs山eLBV. This rninimum valuc idcntifies the row which is aIso the

pivot row. In this row， the column， which contains the unity value of白巴 someunit

vector (or a current basic variable)， is thc column of白cIeaving basic variable.百lis

is theLBV. InTable3.1曲eEBV has already been identified as Xl' The ratios under this column

are 21.25， 8.3333， and 23.3333. The least value is 8.3333.百lIscorr邸 pondsto由e

S巴condrow of Table 3.1. The unit vector under the X4 column is in出isrow (row 2).

Thcrefore， X4 is由eLBV加 d出esecond row is the pivot row for Table 3.2. This

impIies出創出巴 se氾ondrow will be utilized to chang巴出巴 currentXI column (EBV

column) iilto a unit vector白alis currently under X4・Thismechanical procedure is

based on the following reasoning.百levalue ofEBV must be such that it will not cause constraint violation with the current values of血eother basic feasible variablcs. The

application of the Gauss-J ordan process should shift the column under the LBV to出e

EBV leaving the column of the other unit vectors (that are part of the basis) unalt巴red.

Table 3.2 is then constructed using thc pivot row 2.

Simplex Table 3.2 Table3.2

b

nu--nunυ

5.1667 8.3333

45 13530

Xs

nunUE且

nu

X4

-0.1333 0.3333

330

.t)

-且

nununu

Xl

0.7333 -0.3333 7

-1230

XI

121

next

A(:，6)/A(:，1) ans

o 0 0 -17/1980

o 0 0 -51198

o 0 0 -7/99 000 ・16/3

'something not right -The above division should have'; 'been an elemenヒbyelement one' i


ヒhecreate to row pivoヒthe identifies 'The row table' ;

A(:，6) ./A(:，l) ans =

21. 2500 8.3333

23.3333 -5.3333

'Second row is the pivot row and x4 'Constructing Table 2'; 'note the scaling facヒorfor the matrix' i

A

A

1.0e+003 * 0.0004 0.0006 0.0030 -0.0010 0.0030 0.0060

-0.9900 -0.9000 'The element at A(2，1) A(2，:) = A(2， :)/A(2，1)

A

1.0e+003 0.0004 0.0010 0.0030

-0.9900

A(:，6) ./A(:，l) ans

85/4 25/3 70/3

-16/3

format short

is LBV' ;

0.0085 0.0250 0.0700 5.2800

o o

0.0010 o

0.0010 0 o 0.0010 o 0

o 0

must be 1';

0.0085 0.00自30.0700 5.2800

。。0.0010

0

o 0.0003

o o

0.0010 0

0

0

* 0.0006

-0.0003 0.0060

-0.9000

1l l

-

記

4

u

r

H1U乃4y〈

LlNEAR PROGRAMMING

Table 3.3 does not have佃 ynegative value泊白eobjective function row. A solution has been achieved. The basic variables have the unit vector in their column. The nonbasic variables are set to zero. From Table 3.3 the op白numvalues can b巴

observed邸:

x; = 10.4762， xi = 6.4285. X; = 0.4524，I = -21437.14

Note， the valueofj" +21437.14=Ofrom which th巴solutionforthe objective function (f.) can be obtained.

120

Solutlon Uslng MATLAB

The tables in the previous computation can be generated quite effortlessly using a spre叫sheet.百liswould be accomplished by setting up血.emathematical manipulation in one cell and copying and p脳血gto出eother cells. Since this book also邸 sistsin MATLAB progr創nming，the illus住ationhere will allow佃 opportunityto practice the use of MATLAB for matrix manipulations. The code presented here is 四 pturedusing MATLAB'S diary command. The command requires the name of a白川

file which records all communication wilh MATLAB as well ωits response during叩

interactive session (see help diary). The text file is commented liberally (for your understanding) and using the semicolon at由eend of the str.加gsprevents由eirecho.h出isfile the comments are quo飽ds回ng.The code represents由巴detailsof an actual session. These statements are typed a食er白eMATLAB prompt也 theCommand window. This is therefore an interactive session. Values to白eright of ans = is the echo of result of出eex田 utionof出eMATLAB command. Note: comments need not be typed.

3.3.3

Example 3_l.txt

format compact format rational 'Set up the coefficient matrix and b as matrix A';

-1 0 1 0 25;3 6 0 0 1 70;-990 1 0 0 8.5;3 5280]

0.6 o 0

A=[O.4 -900 0 A

17/2 25 70 5280

canonical form';

o 0

1 0

o 1

o 0

note the column' ;

2/5 3/5 1 3 ・1 0

3 6 0 ・990 -900 0

'This is Table 1 -'EBV is x1 -first

'To find LBV divide last column by coeff. in EBV'i 'column' ; 'Take the minimum of the positive values';

122 LINEAR PROGRAMMING

'The element at A(l，l) must be a 0 '; A(l，:) = A(l，:)ー 0.4*A(2，:)

A

1.0e+003 合。0.0007 0.0010 -0.0001 o 0.0052 0.0010 ー0.0003 o 0.0003 。0.0083 0.0030 0.0060 。 o 0.0010 0.0700

ー0.9900 -0.9000 o o 。5.2800 'Element at A(3，1) must be a zero'; A(3，:) = A(3，:) -A(3，1)*A(2，:)

A

1. Oe+003 * 。0.0007 0.0010 ー0.0001 。0.0052 0.0010 -0.0003 。0.0003 o 0.0083

o 0.0070 。-0.0010 0.0010 0.0450 -0.9900 -0.9000 o o o 5.2800

'Element at A(4，1) must be 0'; A(4，:) A(4，:) -A(4，1)*A(2，:)

A

1.0e+004 * 。0.0001 0.0001 ー0.0000 o 0.0005 0.0001 -0.0000 。0.0000 O 0.0008

o 0.0007 o -0.0001 0.0001 0.0045 。ー0.1230 。0.0330 o 1.3530 'Table 2 complete -and canonical form is present' ; 'Solution is not converged because of A(4，2)'; ， EBV is x2' ;

'Calculation of LBV' ; A(:，6) ./A(:，2) ans

7.0455 ・25.0000

6.4286 -11.0000

'Pivot row is third row and LBV is x5' ans

Pivot row is third row and LBV is x5 'Oops forgot the semicolon'; 'Calcu1ation of LBV' ;

'Construction of Table 3.3 -no echo of calculations'; 'A(3，2) must have value 1'; A(3，:) = A(3，:)/A(3，2);

'A(1，2) must have a value of 0'; A(l，:) A(l，:) -A(1，2)*A(3，:);

3.3 NUMERICAL SOLUTlON-THE SIMPLEX METHOD 123

'A(2，2) must have a value of 0'; A(2，:) = A(2，:) -A(2，2)*A(3，:);

'A(4，2) must have a va1ue of 0'; A(4，:) A(4，:) -A(4，2)*A(3，:);

A

A '" 1.0e+004 *

o 0

0.0001 0 o 0.0001 o 0

format rational A

nunUFO

『，

nu噌ム

nU2d

nunununHE

nununU噌よ

-

-

-

-

nununu内

4

nununup口

nununu

円，

内

U内

U内

U『

4

nunununu

nunununu

nununU凋喝

nununuEJ

nununU司ム

nunununu

-

-

-

-

nunununu

噌

ム

向

ununu

nu

nu

nu

nu

A

『，

R

d

r

f

崎』

7

『，

o

'''，，，'''au

噌

ム

円

4

可

-anu

-

-

司

ム

司

4nununu

nunU噌

4nu

nu司

4nunu

-11/105 19/42 1/21 220/21 1/7 45/7 1230/7 150060/7

'No further iterations necessary'; diary off

The last few commands sugg巴slIhc possibility of developing a for loop for the row manipulations. It is suggested as a problem at the end of出echapter.

3.3.4 Solutlon Using MATLA8'S Optimiza舗onToolbox

This section is useful for thosc that have access 10 the Optimization Toolbox. from MA'τ1.AB (Ver 5.2).

Start MATI..AB. In the Command window type help lp. This will provide information on the use of the linear programming routine 10 solve the problem. Here. the standard use is employed. It involves specifying va1ues for the cosl coefficients (ιvector). the coefficientlconstrainl matrix (A-ma凶x).and the right-hand side vector (b・vector).τbe following sequence of steps illus住ates出euse of the program.

Using the Linear Programming Routine

>> f ・990;・900]; >>A [0.4 0.6 3 -1; 3 6)

>>b [8.5 25 70)' >> x '" 1p(f，A，b)

The solution as posled in the Command window:

x 10.4762

6.4286

>> sol '" f'*x


801

・1.6157e+004

To this solution must be added白巴 constant-5280 which was omitted in problem

definition for MATLAB. There ar巴manydifferent ways to use山巴 linearprogramming

function in MATLAB.


h 出issection additional examples釘ep偲 sented.These examples illustrate山eextension/modification of the Simplex method to handle greater由回 orequal

constraints， negative values for the design variables， equalily constraints， and so on. In alI由ecases， the problem is transformed appropriately and白esame Simplex

melhod is then applied.

3.4.1 Example 3.2-Transportation Problem

The Fresh MiIk c∞perative supp1ies miIk in gallon jugs from its two warehouses located in Buffalo (New York) and WilIiamsport (Pennsylvania).1t hぉ acapacity of 2000 gallons per day at Buffalo and 1600 gaIlons per day at Wi11iamsport. It delivers 800 gallons/dayωRochesωr (New York). Syracuse (New York) r，叫uires1440

gallonslday， and曲eremainder (1360 ga1lons) are trucked to New York City.百lecost to ship lhe milk to each of lhe destinations is different and is given in Table 3.4. Estab1ish出eshipping s位置tegyfor minimum ∞st.

Problem Formulat/on: Let x be出enumber of gaIlons shipped from Buffalo to Rochester. Let y be出enumber of gallons shipped針。mBuffalo to Syracuse.

The warehousing cons回 intat BuffaIo is 2∞o gallons/day. Therefore，

x + y$ 2000

Amount shipped企omWilliamsport to R，∞hes町 =8oo-xAmount shipped企omWilliamsport to Syracuse = 1440 -y Amount shipped from Williamsport to New York City is

1600 -(800 -x)ー(1判O-y)注o

Table 3.4 Shipping Cost (Cen匂 perGallon)

Buffa10

WiIli創nsport

Rochester

4.2 4.7

Syracuse

4.5 4.6

(3.21)

New York City

6.0 5.1


x+y~ 64O (3.22)

The side constraints on x and y are th巴respectivewarehouse Iimits. The shipping cosl IS

Cost=4.2ホx+ 4.5 * y + 6.0 * (2000 -x -y) +

4.7・(8oo-x)+4.れ(1判0-y) + 5.1 ・ (x + y -640)

Cost=ー1.4'"x-y + 19120

Assemb1ing the problem (Example 3.2)

M泊b凶ze f(x，y):ー1.4x-y+19120

Subjectω81(X， y): x + y :5 2000

82(X， y): x + y注640

0$x:58∞; 0:5y:51440

(3.23)

(3.24)

(3.25)

(3.26)

(3.27)

Figure 3.7 iIIustrates the graphical solution to Example 3.2. The line representing the constant objective function values lerm does not include the constant term (19120).

The solution is at the interseclion of constr温intgl and the upper 1imit on出evalue of

xas出eobjective is to decreasefas much as possible. From the figure the solution is

x=800;y= 1200，f= 168ooor$168.oo (3.28)

Two-Phase S/mplex Method: The main difference belween Examples 3.1 and 3.2 is出e注cons町aintin the Iatter. Since the standard LP problem only requires the design variables be semipositive位 0)，出eright-hand constraints on x and y have to be accommodated through additionaI inequaIity constraints. Applying the regular Simplex method without additionaI processing would cr回 tea problem in recognizing 也ec釦 oni回 1form. Example 3.2 expressed in出estandard format of linear programmmg IS

Minimize ft.x，y):ー1.4x-y + 19120

Subject to: gl(X， y): x + y + SI = 20∞ g2(X， y): x + y -S2 = 640

g3(X， y): x + S) = 800

84(x， y): Y + S4 = 1440

(3.24)

(3.29)

(3.30)

(3.31)

(3.32)


25∞

ωgo=ω回Hhh

句』司

崎町

』司

• 』『

』司

』司

~脚。2 2 ~ 500 E 。

D

-司』

司

』

司

』

『

』

司

句

句‘

‘

句

』

司

』

司

句

--

‘

』

『h

-『

』司

』司

』司

』司

』-h

• ‘

』『• 』『‘

』司司

-5ω -・‘』旬。

-1000

800 -1500

0 200 300 400 500 600 x:g副10間 110mB凶f副oloR∞h田101

100 700

Flgure 3.7 Graphlcal 80lullon， Exampla 3.2.

x ~ 0; y ~ 0; sl' s2' s3' s4 ~ 0 (3.33)

Heresl・S3's4 are the slack variables and S2 is白巴surpl.凶 (similarto a slack-used for

~ constraints) variable. In Example 3.1， when the f1rst Simplex table was set up (Simplex Table 3.1)， the slack variables were helpful in identifying the canonical

form. Here in Example 3.2， it do凶 notworkout伽 tway because the coefflcient of S2 is -1. Multiplying Equation (3.30) by -1 migrates the negative sign to the right-hand side. which is also disaJlowed as f:紅 as白erecognition of白ecanonical form is

concerned. This means additional problem-specific preprocessing must take place to

identify the initial canonical form. Since the Simplex me出odis used to handle large

problems with several variables， it is more convenient to apply the Simplex procedu陀

in a consistent way. This is accomplished by applying the same Simplex technique in

a two-part sequence.

百lef1rst p副 isrecogt甘zedas Phase 1. Here a new kind of variable. an artificial variable， is defined for each surplus variable in the same equation. Also， a new

objective function called an artificial cost functioll or加 artificialobjective function is introduced. The artiflcial objective is defined邸 asum of aJl the artificial variables

in the problem. In Phase 1，白eartiflcial objecti ve is同 ducedto zero using the standard

Simplcx procedure. When this is accomplished， Phase 1 is complete. If山eartificiaJ

i40


objective depends only on the artificiaJ variables.佃 dif its vaJue is zero，出isimplies

that白巴 artificiaJvariables are lIonbasic variables. This also suggests that these

variables werc basic variables at the start of the procedurc. When Phase 1 is completed，曲enboth the artificiaJ obj巴ctivefunction and the artiflcial variables are discarded台。m

出巴tableand Phase 1/ begins. Phase II is the standard Simplex technique applied to the table from the end of

Phase 1 neglecting all山 artificialelements. The table should be in canonical form.

Additional tables are obtained as necessary until the solution is reached. In Example 3.2 set up earlier， there will be one artificiaJ variable a 1 and an artificiaJ

cost function A" In the example only Equation (3.30) will be affected as

g2(X，y): x+ y-s2 + al =640 (3.34)

The artificiaJ cost function will be

Af=al (3.35)

Table 3.5 rep陀 sents出巴 f1fSttable in Phase I. The f1rst four rows rep隠 sent白E

constraints. The f1fth row問p民sents血eoriginal obj田町efunction. The last row is由eartificiaJ objective function. This is出巴 rowuSed to合iv巴出巴 iterationsin Phase 1. Table 3.5 is not in canonicaJ form. but replacing the sixth row by the result of

subtracting血esecond row from the six出 rowwill provide a canonical form. Table 3.6 illustrates the result of such a row manipulation.

SlmplexMe的od，PhBse 1: In Table 3.6 there are two choices (x， y) available for

曲eEBV(ente自Igbasic variable) as both of them have a coefficient of -1 in血elast

row. While the choice can be arbitrary， x is a good choice because it has a larger negative coefflcient in the originaJ objective function.百leLBV (leaving basic

variable) is identified through the minimum positive vaJue of the ratio of the values in

the b column to出evalues under出巴xcolumn.官lescratios are 2000/1，64011， and 80011. The selection identifies出esecond row as自己pivotrow and a 1 総白eLBV.Using the second rowω 血epivot row， the unit vector [0 1 0 0 0 O]T hωto be

Table 3.5 Example 3.2: Initial Table， Phase 1

x y SI S2 01 S3 S4 b

o o o 。 2000 。。。 640 l 。。。。 o 800 。。。。。 1440

ー1.4 。 o 。。。f-19120

。 o 。。。。 Af


Table 3.6 Example 3.2: Slmplex Table 1， Phnse ， (Canonl凶 IForm)

x $1 $2 b al $) $4 y

O

-1.4

1

l

u

l

l

--一

-shununuhunu

。↓。。。 o f-19120

o Af-640

nuhunu'且

2000 640 800 1440

AU'anu白

ununu

hunu-anunuhu

町創lsfe町edfrom the al column to the x column. Construction ofTable 3.7 st訂tswith the second row in Table 3.6， which is also the second row ofTable 3.7 because ofthe coe仔icientof 1 in the second row under the x column (no scaling is necessary). This is the pivot row. To obtain a 0 in the first row in the x column， the pivot row is subtrョctedfrom出巴白rstrow in Table 3.6. The new first row is

。。 -1 0 0 1360

To obtain a 0 in the x column in the third row. the pivot row is subtracted from the third row ofTable 3.6. The fourth row has a 0 in place and hencc is copied合'OmTable3.6. Thc fifth row is obtaincd by adding a 1.4 multiple of the pivot row to the fit1出rowin Table 3.6. The last row is the addition of the pivot row to the last row from Table 3.6. Table 3.7 is compiled as follows:

Table 3.7 Example 3.2: Slmple宜Table2，Ph脚 1(Canonlcol Form)

x $2 b $) $1 al $4 y

nu'且

AUnununu

凸

unu'anununu

o f-18224

o Af

nunuhu'a

1360 640 160

1440

aaマ

0

1」1

0

0

'anununununu

O

-1.4 組品守

'a'E'anu-a

--

From Table 3.7.山evalue of Af is O. and al is not a basic variable・Thisidentifies the end of Phase 1. The a 1 column and白巴lastrow are discarded and Phase 11 is started with Table 3.8.

b

V

L

F

i

j

i

E

-

E

圃

|


Table 3.8 Exumple 3.2:創mplexTable 1， Phase s (CanoniωI Form)

x $4 b y $2 $) $，白

u'a凸

UnunU

nununu'a

1360 制0160

1440

0

0.4

-EAUAununu

nunυ'anunu

o -1.4 o f-18224

Simplex Method， Phase 11: The column with the gr田 testnegative coefficient (一1.4)is s2・百lisis出eEBV. The LBV is s3・百le出irdrow is therefore the pivot row. The unit vector [0 0 1 0 O]T must be町ansferredfrom the S3 column to由巳 S2column through row manipulations using the pivot row.百首sexercise results in Table 3.9.

The negative coefficient under the y column suggests we are not fmished yet. For the next iteration， EBV is y. LBV is SI' The first row is出epivot row. The unit vector [1 0 0 0 0 ]T must be constructed under the y column using row manipulation using the pivot row. This results in Table 3.10.

There are no negative coefficients in the last row. The solution has been rellched. From inspection of Table 3.10， the solution is

x = 800， Y = 1200. S2 = 1360. S4 = 240，f= 16800 (3.36)

In this ex創nple，the same Simplex method was repeatedly applied although the problem was solved using抑 ophases. The first phase was a preprocessing phぉeωmove the surplus variables away from出巴 initialset of basic variables.百liswas achieved by introducing additionnl variables for each su叩lusvariable and an additional cost function that drove出eiterations of the frr宮tphase.

To summarize， for ~ and = constraints， artificial variables and釦 artificialcost function釘ein官。du回 dinto the problem for convenience.百leSimplex method is applied in two phases.百lefirst phase is terminated when the artificial variables clln be eliminated from山巴problem.

Table 3.9 Example 3.2: Slmplex Table 2， Phas量n(Canonlcal Form)

x $) $4 b y $， $2

白

u・且

AUAUnu

0

1.4

o 12∞ o 8∞ o 160

1440

o f-180ω

凸

unu'anunu

'anusaE且

Ea

--

'Enunununu

3，A

直組長3

130 UNEAR PROGRAMMING

Table 3.10 Example 3.2: Slmplex Table 3 (Fioal)， Phasc D

x y SI S2 S) S4 b

。。。 12叩。。。 1 。 800 。。 o 。 1360 。。。 240

。。。 0.4 。f-16800

3.4.2 Example 3ふ-EqualltyConst悶 intsand Unrestricted Varlables

Example 3.3 wil1 inc1ude an equality constraint and iJlustrates the technique for handling variables白紙areunres町ictedin sign (can have negative values). Variables

like profit， temperature， and net income can be negative quite ofl刷 1.The叫uality

cons回 intis handled by in凶 ducing印刷ificialvariable and applying出etwo-phaseSimplex tl田加iqu巴 illus回 tedin Example 3.2. When this correspondi略的自cial

variable is zero (巴ndof Phase 1)，由巳constrainthas been met. The negative value for a variable is simulated through出ediffl巴rencebetween two positive variables. Once

again， these chang巴spreserve the Simplex method in its original form. The following

is an imaginary problem ωillustrate additional features so some Liberty is taken in the formulation.

The Problem: Today， a full-tirne student on campus is always driven to maximize

grades. A strong influence for a favorable outcome is血eamount of inves回開tmade

in hitting出巴booksversus出etime spent playing pinball. In a typical day， at least one hour is definitely extended to the pursuit of learning or pressing the sides of the

coin-operated machine. Not more than five hours is availabl巴 forsuch dispersion.

Over出eye紅 s，a fatigue threshold of four units. based on a combination of白巴 two

activities， has been established as a norm for acceptable perform四 ce.This

combination is the sum of the hours spent at the pinball machine and twic巴thehours

spent on homework， an acknowledgment that hitting血ebooks is stressful. A negative

hour of pinball playing wi1l go to increase the time spent studying， which will contribute positively to the grades. The overall grade is determined as a linear

combination of twice the time spent academically and subtracting白etime spent

playing. The problem出ereforeis to distribute the time to maximize the grades

obtained.

Problem Formulation: There are two original design variables in the problem. XI is the number of hours spent studying， and X2 is the time spent enjoying the game of pinball when th巴sametime can be advantageously spent maintaining high grades.τ'he

obj配 tivefunction can be expressed as

3.4 ADOITIONAl EXAMPLES 131

Maximize j{xl， X2): 2x1ーら

The constraints can be recognized from the statement of出eproblem directly as

g.: xl+ Xz ~5

g2: 2x. + x2 = 4

g3: x1 +X2~ 1

x1 ~ 0， x2 is unrestricted in sign

Figure 3.8， using the drawLine.m m-file， is a graphical description of the problem and

the solution.明1esolution is atxl = 3 and X2 = -2 in自己figure，where conveniently白eobjective function is also passing through.

Standard Format: As the standard format expe出 onlynonnegative variables. the

variable x2 is replaced by a pair of variables in the formulation:

6

1

0

・0o，a

Zト¥;¥ ドーく!

4 5 6

Flgure 3.8 Graphlcal solution， Example 3.3.


x2 =x21 -X22 (3.37)

The standard fomlat after converting to a mininuzation problem， introducing slack， surplus，佃dartificial variables， and including Equation (3.37) results in


Phase " Table 2: Table 3.13 is 1he second table under Phase 1. The vaJue of the artificial function is 2， so Phase 1 is not yet over. The EBV is X22・TheLBVisal・Thesecond row is 1he pivot row. This row wiII be used for row manipulations ωyield the next table.

Min回rize f -2x1 +x21 -X22 (3.38) Table 3.13 Example 3.3: Tablc 2， Phasc I 響|

Su険jectto: gl:X1 +X21-X22 +81 =5 X22 $) U) 82 a2 b X) X2)

gl: 2x1 +X21-X辺+at=4 。。。 4 。g3: X1 + X21 -X22 -82 + a2 = 1 。。 2 -2 2 。。 l

The artificiaJ function to be driven to zero is 。 3 -3 。。 -2 2 /+2 (3.39) 。。。 -2 3 Ar-2 Af=al+~

Simplex Table 0: Table 3.11伺p伽悶1hes回 dardfOm18t in a table. It is caIIed Table o to aIIow for preprocessing wi1h respectωthe artificia1 variables so that a canonica1 fOlID 四nbeob鴎rved.Adding血esecond卸 d血irdrow阻 dsubtrac出g血eresult合om由elast row and replacing the last row wi1h this computation wil1 yield the next table.

Phase " Table 3: Table 3.14 is the new table after the row operations. The value of 由巳 artificiaJfunc丘onis 0 as bo1h al and a2釘 enonbasic variables with a vaJue of 0 for白isiteration.百lIssig凶fies白eend of Phase 1. Phase n will start with the last row ( artificia1釦nction)and the two columns that represent the arti白cialvariables removed from the current table.

Tablc 3.11 Examplc 3.3: Table 0， Phasc I T自blc3.14 Ex圃mplc3.3: Table 3， Phase Iσ10111)

8) a) S2 a2 b 82 oz b

X) X2) X22 X) X2) X22 $) 。1

1 。。。 5 l o 4 。。 4 。 o 。2 I 。。。 2 一2 2 o 。 -1 1 。 3

1 。 o -2 I 。。。。 f 。 o 。 3 4 4 /+8 。 Ar 。。。 o 。 l 。。。。。 1 Ar

Phase 1， Table 1: Table 3.12 provides白estart of由eSimplex method. In Phase 1 th巴motivationis to drive the artificial function to 0， which has a value of 5. The EBV iSXl・TheLBV is a2' The third row is the pivot row. Row manipulations with the pivot row lead to Table 3.13.

Tllblc 3.12 Examplc 3.3: Table 1， Ph由scl

X， X2' X22 $， a) $2 a2 b

1 。 o 。 5 。 o 。 4 o 。

-1 。。。。 f 2 。。。Ar-5

'Aqι

1

2

1

2

3

--

Phase 11， Table 1: Table 3.15 represents曲etabl巴tostart the itemtions for Phase 11. However， there are no negative coefficients in曲巴lastrow (row corresponding to the objective function). Scanning the帥 lefu氏her，1he canonical form can be observed. 百lerefore，the solution has been obtained.百lesolution is

Tablc 3.15 Examplc 3.3: Table 1， Phasc 11

x， X幻局z $， 82 b

。。。 4 。。 2 2 o o 。 3

。。。。 4 /+8

134 LJNEAR PROGRAMMING

xl = 3;x21 =0;x22 = 2; sl =4; s2 =0;/=-8

The value of X2 is

X2 = (X21-x22) = (0-2) =-2

which was identified earlier graphically. 百leabove example illustrated the Simplex method for handling negative variables

and equality constraints. Not巴自己 application of the Simplex technique itself did not change. An interesting feature of白eprobJem was that Phase II was immediat巴lyobtained aftel' Phase 1 without any further iteration， which suggests白紙thesearch for the optimum is taking place in Phas巴1even though出efocus is on the artificial variables.

3.4.3 Example 3.4-A Four-Varlable Problem

Example 3.4 presents a problem with four variables. The procedure is identical to the one in Example 3.1， except出at血ereis no graphical solution to the problem.

The Problem: The RIT student-run microelectronic fabrication facility is taking orders for four indigenously developed ASIC chips出atcan be us吋 in(1) touch sensors， (2) LCD， (3) pressure sensors， and (4) controllers. There are several constraints on the production based on space， equipment availability， student hours， and the fact山at出eprimary mission of白efacility is student training. First，出ehandling time cons回 int， outside of processing， for all chips is 600 hours. Touch sensors require 4 ho町 s，LCD 9 hours， press町 'esensors 7 hours， and con住ollers10 hours. Second， the time available on the lithographic machines is about 420 hours. Touch sensors and LCD r叫uire1 hour， pressure sensors 3 hours， and controllers 8 hours. Packaging considerations place the maximum at 800 volume units. Touch sensors require 30 voJurne units， LCD 40 volume units， press町'csensors 20 units， and con住oller百 10units because of山eircompact size. All山econs住aintsabove are indi回 tedper week of operation of the facility.百 enet revenue is $6， $10， $9，叩d$20for the touch sensor， LCD， p問ssuresensor， and controller， respectively.百lefacility is interested in maximizing revenue per week and would ¥ike to determine the right mixof由巳fourdevices.

Problem Formulation: The formulation is straightforward based on the statements above. Let x( rep問sentthe number of touch sensor chips per w田 k，X2白Enumber of LCD， x3 the number of pressure sensors， and X4白enumber of con佐ollers.明記 objectivefunction is

Maximize: f. 6x. + lOx2 + 9x3 + 20x4

The handling time constraint can be expressedω


gl: 4xI + 9x2 + 7x3 + IOx4 S 600

The availabi1ity of the lithographic machine can be developed as

g2: xl + x2 + 3x3 + 8x4 s; 420

百Jepackaging constraint is

g3: 30xI + 40.ら+20x3 + IOx4 S 800

All design variables ar官邸p田 tedto be g偲 ater白肌 zero.As formulated above，出eproblem suggests a degree of incompleteness.τnere arefour design variables and only three constraints.明lenumber of variables in the basis can only be山間e.Hence， at least one of出evariables must have a value of zero. Sev巴raluseful additional constraints can stiU be included to define a valid optimization problem with a nonzero solution.前lisis now a modeling issue. For problems with a limited number of design variables， paying attention 10 the problem development allows anticipation of the solution as well俗 theopportunity to町oubleshootdecisions from practical considerations. It is left to出estudent to explo陀 thisproblem further.

Standard Format: The objectiv巴 functionrequires a minimum formulation. Henc巴，

Minimize f. -6x1 -1O~ -9x3 -20x4

The symbol / has been retained for convenience even though the dII宅ctionof optimization has changed.百lecons回 intsare set up using slack variables SI・S2，ands3・

g.: 4xI + 9x2 + 7x3 + 10x4 + S. = 600

g2: x. + ~ + 3x3 + 8x4 + s2 = 420

g3: 30x. + 40x2 + 20x3 + lOx4 + S3 = 800

All variables are ~ O.百leformulation above suggests the standard Simplex method.

S何'P1exTable 1: Table3.16is出eini削 tablefor Example 3.4. The canonicaI form is observable from the table and the initial basic feasible solution can be determined.

Table 3.16 Examplc 3.4: Simplex Table 1

XI X2 XJ X4 81 J"2 SJ b

4 9 7 10 。。6∞ 3 8 。。420

30 40 20 10 。。 800

-6 -10 -9 -20 。。。f


Table 3.17 Example 3.4: Simplex Table 2

x， X2 X3 X4 s， 82 8J b

2.75 7.75 3.25 。 l -1.25 。 75 0.125 0.125 0.375 。 0.125 。 52.5 28.75 38.75 16.25 。。ー1.25 275

-3.5 -7.5 -1.5 。。 2.5 。f+ 1050

SI・S2.and $3 are the basic variables. The EBV is X4 and白eLBV is $2 which is白Emin泊lumof {60. 52.5， 80}. The pivot row is the second row used for the row manipulations to lead to Table 3.17.

Simplex Table 2: Using the pivot row identified in山elast table，出eunil vector [0 o 1 O]T under the $2 column needs to be transferred to出eX4 column through elementary row operations. Table 3.17 shows白ecanonical form after completion of all the operations. The basic variables are X4. $"組 dS3・TheEBV is X2 and白eLBVis S3・百lepivot row is the白irdrow. The objective of the row manipulations is to transfer the unit vector [0 0 1 O]T from the $3 column to白exzcolumn.

Simplex Table 3: Table 3.18 denotes the reduced table wi出thecanonicaJ form after the required row operations are completed. The basis variabl回釘eX2.x4，and SI・Therem芭nonegative coefficients in the last row suggesting the solution has been obtained.

From白etable白巴solutionis

XJ = 0， x2 = 7.096， x3 = 0， x4 = 51.61，/= 1103.22

The solution above is not satisfactory because the actual decision wiIl involve integer values for the design variables. At this time integer programming is not an option. The adjusted solution is

X1 = O，x2 = 7，x3 = O，X4 = 52，/= 1110

While白ischoice satisfies出econstraints g3. g2 however needs a Iittle elasticity (limit is 423) to be satisfied. Not巴白atsJ is a basic variable so constraint gJ should not be a

problem.

Table 3.18 Example 3.4: Simplcx Tnble 3

x， X2 XJ X4 $， 82 $) b

-3 。。。 -0.2 20 0.032258 0 0.322581 o 0.1290322 -0.00323 51.6129 0.741935 1 0.419355 。。-0.032258 0.025806 7.096774

2.064516 0 1.645161 。。2.25806451 0.193548 f+ 1103.22


Using MATLAB

This is a standard LP problem. This section uses MATLAB in an interactive session to soIve白eproblem.百lefollowing is a diary file for the MATLAB session for出isex創nple.Note the waming from MATイLAB.as weIl凶白esubsequent use of the LP program using bounds on曲edesign variables:百lebold italicized statements are commands used to obtain the solution. The MATLAB prompt is edited out.

help lp

LP Linear programming. X=LP(f，A，b) solves the linear programming problem:

min f 'x subject to: Ax <= b X=LP(f，A，b，VLB，VUB) defines a set of lower and upper bounds on the design variables， X， so that the solution is always in the range VLB <= X <= VUB. X=LP(f，A，b，VLB，VUB，XO) sets the initial starting point to XO.

X=LP(f，A，b，VLB，VUB，XO，N) indicates that the first N

constraints defined by A and b are equality constraints.

X=LP(f，A，b，VLB，VUB，XO，N，DISPLAY) controls the level of warning messages displayed. Warning messages can be turned off with DISPLAY -1. [x，LAMBDAJ= LP(f，A，b) returns the seヒofLagrangian multipliers， LAMBDA， at the solution.

[X，LAMBDA，HOW] LP(f，A，b) also returns a string how that indicates error conditions at the final iteration. LP produces warning messages when the solution is either unbounded or infeasible. c = [-6 -10 -9 -20]; A = [4 9 7 10; 113 8; 30 40 20 10]; b = [600 420 800]; x = [c， A， b] x = lp(c， A， b) Warning: The solution is unbounded and at infinity; the constraints are not restricヒiveenough. x 1.0e+015 * -0.0000 3.7067 ・8.83912.8513


vlb = [0 0

vub = [100， x = lp(c， A， x 0.0000 7.0968 o

51.6129

0] ;

100， 100]; b， vlb， vub)

This is thc solution obtained by working through the Simplex method.

3.5 ADDITIONAL TOPICS IN LlNEAR PROGRAMMING

This section looks briefly at additionaJ ideas associated wi山 theLP problem. First，出ereis a discussion of the dual problem associated with an LP problem. Duality is impo此antin恥 discussionof sensitivity analysis-var削 ionin optimization solution due to variation in tfte origil聞 1parameters of tlte problem. Following出isis a

discussion of several pieces of inform凶onthat can be ex回 ctedfrom出efinaJ table of the Simplex method.

3.5.1 Prlmal and Dual Problem

Associated wi出 everyLP problem， which will be referred toωa primal problem， there is a corresponding di即 1problem. In m佃 yways，出eduaJ pl'Oblem is a

町叩spositionof the primaJ probJem.百lIsis meant to impJy that if the primal probJem is a minimization probJem白eduaJ problem will be a maximization one. lf the primaJ problem has 11 variables， the dual problem will have n constraints. lf the primal problem has m constraints， the duaJ problem will have m variables. For the discussion of duality，出estandard primCll problem is generally defined as a maximization problem with ~ in叫ualities(though白estandard problem in the LP programming discussion was a minimization problem [s田Equations(3.10)-(3.12)]加 d由e∞ns回 Intswere叫ualities).The s凶伽rdprimaJ problem in this section is defined as [3]

Maximize z: CIXI + CIX2 + . . . + c，，x，， (3.40)

Subject 10: allxl + al2x2 + . . . + al，.xn ~ b， (3.41)

a21xI + a2~2 + . . . + az"xn ~ b2

a"， IXI + anaX2 +・・・ +an"，xn ~bm

与さO，j=1，2.....n (3.42)

戸薬

:;

還

3.5 ADDITIONAL TOPICS IN LlNEAR PROGRAMMING 139

This is also referred 10踊 anormal maximum problem. 百ledual of the above problem is defined as

Minimize w: blYI + blY2 + . . . + b"J'rn (3.43)

Subject 10: allYI + a21Y2 +. . . + arnlYm ~ cJ (3.44)

al2Y1 + a22Y2 + . . . + am2Ym ~ c2

al，J'1 ... a2"Y2 + . . . + a，tuJ'"，さC"

YI~O， i= 1， 2，... ，m (3.45)

Relations (3.43)ー(3.45)describe a normal minimum problem. There is also an inverse relationship between曲巴 definitionof the two problems above as far as山巴

identification of the primal and dual problem. If出巴 latteris considered a primal

problem，出enthe former is出ecorresponding dual problem.

百lefollowing observations c回 bemade wi由民spect10 the pair of problems established above:

1. The number of dual variables is出巴 sameω出enumber of primal constraints.

2. The number of dual constraints is thc same as the number of primal variables.

3. The coefficient matrix A of the primal problem is transposed to provide山E

coefficient matrix of the dual problem.

4. Thein叫ualities釘'Creversed in direction.

5. The maximization probJem of the primaJ problem becomes a minimization

probJem in th巴dualproblem.

6. The cost coefficients of thc primal problem become the right-hand side vaJues

of the duaJ pl'Oblem. The right-hand side vaJues of the primal become the cost

coefficients of the dual problem.

7. The primal and duaJ variables both sa出fythe nonnegativity condition.

Example 3.5百lefollowing example， adapted and modified from Reference 3， is used to iIIustrate the primal/dual versions ofthe same problem. The problem is solved using techniques developed in this chapter.

而 eProblem: The local college's School of American Craftsman has decided 10 participate seriously in a 10caJ charity event by manufacturing special commemorative

desks， tables， and chairs. Each 1ype of fumiture requi陀 slumber as weIl as two types

of skilled labor: fmishing加 dc細'Pentry.Table 3.19 documents the resources needed.

The 10caJ lumber cooperative has donated 48 board feet of lumb巴r.Faculty and staff

have volunt田町d20 finishing hours and 8 carpentry hours. The desk will be sold for

140 llNEAR PROGAAMM1NG

Tnblc 3.19 Example 3.5: Rωources

Resources Desk Tabl巴 Chair

Lumber(bo釘dfeet} 8 6 Finishing hours (hours) 4 2 1.5

Carpentry (hours) 2 1.5 0.5

$60，出etable for $30， and出巴chairfor $20. The school does not expect to seJ1 more

出印日vetables. The school would like to maximiz巴rev巴nuecoJ1ected.

8tandard Formaト PrimalProblem: Defining Xl as the number of desks

produced， X2 as the number of tables manufactured， and X3 as the number of chairs

manufactured， the primal problem can be expressed in standard format悩

Maximize z: 60x， + 30X2 + 20X3

Su句ectto: g， : 8x， + 6x2 + X3 :::; 48

g2: 4x， + 2x2 + 1.5X3 :::; 20

g3: 2x， + 1.5x2 + 0.5X3 ::; 8

1!4: X2::; 5

Xl>X2 ，x3 ~O

(3.46)

(3.47a)

(3.47b)

(3.47c)

(3.47d)

The solution to the primaJ problem is avai1able in Tables 3.20-3.22. Table 3.20 is th巴initiaJ table. The EBV is x， and the LBV is $3・Thepivot row is the出irdrow. The

subsequent row operations determine Table 3.21. In this table，出eEBVisx3'theLBV is sz，zmd the pivot row is the SEcond Eow.Table 322is the anal table establishing血E

optimal values for the variables. xt，白eoptimaJ number of desks， equals 2. ~，血e

Table 3.20 Exnmple 3.5: Primal Problcm， Simplex Table 1

x， X2 X) $， $2 S) $4 b

B 6 。。 o 48

4 2 1.5 。。 o 20

2 1.5 0.5 。。。 8 。。。。。 5

-60 -30 -20 。。。。 f+O

l lil 1

「主

d d d f

S 堅

l準1

最E

3.5 ADD1T10NAl TOP1CS IN UNEAA PAOGAAMM1NG 141

Tablc 3.21 Examplc 3.5: Prbnal Problcm， Simplcx Tablc 2

X， X2 X3 $， $2 $3 $4 b

。。 1 。 4 o 16 。 0.5 。 -2 。 4 0.75 0.25 。。 0.5 。 4 。。。。。 5

o 15 -5 。。 30 。 f+240

optimal number of chairs to be made， is 8. No tables will be made.百letotaJ revenue

is$280.00.

81.ヨndardFormat ー DualProblem: While the duaJ problem can b巴巴xpressed佃 d

setup m田 h阻 ica1ly，it is 函館cultto associate出edesign variables in a direct m佃 ner

aswasdone泊 theprimaJ problem. Using y" Y2， Y3， and Y4 as the design variables， the problem can be formulated as in Equations (3.48) and (3.49) below using the

definition at由ebeginning of this section. Some associations are possible in light of

transposition of出ecost coefficients and the constraints. For example， the variable y， can be associated with the lumber constraint because in the dual problem its cost

coefficient is the lumber constraint of出eprimal problem. In a similar manner， Y2 is

associated with the finishing hours， Y3 with出巴carpentryhours， and Y4 wilh the speciaJ

table constraint.明記objectivefunction of the dual problem is expressed in Equation

(3.48). In view of白eform of the expression of the objective function w，血E毘 isa

possibility that出edesi伊 V紅iabl民 inthe duaJ problem can have四 econornic

implication as fol1ows:

y， unit price for a board foot of lumber

Y2 unit price for a finishing hour

Y3 unit price for an hour of carp巴ntryy4 白iscannotbe白eunit price as也edesk has a price defmed in the primaJ

problem-maybe a speciaJ price for the desk resource

Table 3.22 Example 3.5: Primal Problem， Simplex Table 3

X， X2 X3 $， $2 .1'3 $4 b

。 -2 。 2 -8 o 24 o -2 。 2 4 。 8

1.25 。。 -0.5 1.5 。 2 o o o 。。 1 5

。 5 o o 10 10 o f+280


The dual can be expressed as

Minimize w: 48YI + 20Y2 + 8Y3 + 5Y4 (3.48)

Subject to: hl: 8YI + 4Y2 + 2Y3 + OY4注60 (3.49a)

h2: 6YI + 2Y2 + 1.5Y3 + lY4 ~ 30 (3.49b)

h3: lYI + 1.5Y2 + 0.5Y3 + OY4 ~ 20 (3.49c)

YI> Y2， Y3 ~ 0 (3.50)

lntroducing surplus variables (SI> S2， S3) and artificial variabl巴s(al・。2'a3)出edual problem is expressed as a standard LP problem:

Minirnize w: 48YI + 20Y2 + 8Y3 + 5Y4 (3.48)

Subjectto: hl: 8Yl + 4Y2+ 2Y3 -SI + al = 60 (3.51a)

h2: 6)'1 + 2Y2 + 1.5Y3 + lY4 -S2 + a2 = 30 (3.5Ib)

h3: lYl + 1.5Y2 + 0.5Y3 + OY4 -S3 + a3 = 20 (3.51c)

[n Phase 1 of the two-phase approach，山eartificial variables are removed from白E

basis using an ar甘白cialcost function A defined as

MinimizeA: al + a2 + a3 (3.52)

The Phase [ computations are displayed in Tables 3.23-3.26. Table 3.23 is identified as th巴simplexTable 1， and it inc1udes some preproccssing by which a" a2， and a3釘emade山ebasis variables. As a reminder， the last row問presents山earti ficial∞st function. From inspection ofTable 3.23， YI is the EBV. The LBV is a2 and由巳pivotrow is the second row. Following the standard row operations， Table 3.24 is obtained. [n this table， the EBV is Y2 (也elargest negative coefficient).ηle LBV isa3 (白E出ird

Table 3.23 Example 3.5: Dual Problem， Phose 1， Table 1

Y1 Y2 Y3 Y4 51 52 S) “l Q2 a) b

8 4 2 。。。。。60 6 2 1.5 。。。。30

1.5 0.5 。。。。。 20

48 20 8 5 。 o 。。。。 w

-15 ー7.5 ー4 。。o A -110

3.5 AOOITIONAL TOPICS IN LlNEAR PROGRAMMING 143

Table 3.24 Example 3.5: Dual Problem， Pbase 1， Table 2

Y1 Y2 Y) Y4 51 52 S) a1 a2 a) b

。1.3333 0 -1.3331 1.3333 。 -1.3333 。 20 0.3333 0.25 0.1666 o -0.1666 o 。0.1666 。 5

o 1.1666 0.25 -0.1667 。0.1666 -1 o -0.1666 15 。4 4 -3 。8 。o -8 o w-2柑

o -2.5 -0.25 1.5 一1.5 。2.5 。A-35

row contains the rninimum of ([20/1.333]， [510.3333]， [15/1.6667]}).百lepivot row is row 3. Table 3.25 rep隠 sentsth巴nexttable.百leEBViss2・Notein this cぉεthereare two candidates for EBV (S2・S3)'S2 is chosen arbitrarily.百leLBV is al' The pivot row is出E

街百trow. After the [珂uiredrow manipulations， Table 3.26 is generated. Scanning the last row， th釘eare only positive coefficients. Phase 1 is now complete. The artificial variables and白巴紅tificialcost function can be removed from the LP problem.

Elirninating白eartificial variables and the artificial cost function from the problem， Phase II is started with Table 3.27. Not巴thatTable 3.27 contains no information that is not already available in Table 3.26. It only provides an unc1uttered table for continuing th巴 applicationof the Simplex lechnique， and therefore is not really necessary. In Table 3.27， the EBV is S3・TheLBV is Yl after a toss between Yl and sl' The pivot row is the second row. This row is used to obtain Table 3.28. In this table， note that the value of the basic variable SI is O. This is called a degenerate basic variable. The EBV for the next table is Y3・TheLBViss3・Thepivot row is白es田 ondrow (note: the pivot row can be曲erow with the degenerate basic solution if the column of the EBV has a positive coefficient in曲atrow). Table 3.29 is曲efinal table and contains the optimal solution. The final solution of the dual problem is

Y; =0，活=lO，y; = 10，見=0，s;=5， w・=280To summarize， the solution of the primal problem is

x; = 2， xi = 0， x; = 8， s;p = 24，勾p=5，z*=280

・fhesubscript “p" is added to associate the slack variable with白eprimal problem. The solution of the dual problem is

Y; =0，点=10， Y; = 10， Y4 = 0， si = 5， W・=280

Some Formal Observations

These observations are justified morc formall y in many books on linear programming， some of which have been inc1uded in the references at the end of this chapter. Here， the formal results are elaboraled using the solution 10 the example discussed in this

-品晶

Table 3.25 Example 3.5: Dual Problem， Phase 1， Table 3

y， Y2 Y3 Y4 $， $2 S3 a， a2 a3 b

。 o -0.2857 -1.1429 1.1429 1.1429 -1.1429 -l.¥429 2.8571 。 0.1786 0.2143 。-0.2143 02857 。 0.2143 -0.2857 0.7143 。 0.2143 -0.1429 。 0.1429 -0.8571 。-0.1429 0.8571 12.857 。 o -4.8571 ー2.4286 。 7.4286 3.4286 。ー7.4286 -3.4286 w -291.43 。。 0.2857 1.1429 -1.1429 -1.1429 。 2.1429 2.1429 A -2.857

Table 3.26 Example 3.5: Dual Problem， Phase 1， Table 4

y， Y2 Y3 Y4 $， $2 S3 a， ~ a3 b

。 o -0.25 -0.875 0.875 2.4999 。 0.125 -6E-O岳 -0.1875 6E-06 0.5 0.1875 -6Eー凶 -0.5 1.25 。 0.25 4E-05 0.125 -4E-05 -0.125 4E-05 12.5

。。 -3.0001 4.9998 6.4998 0.0002 -3.9998 -6.4998 -0.∞02 3.9998 w-3IO

。。。。 o 。。 A

宇品切


Tab1e 3.27 Exomp1e 3.5: Duo1 Prob1em， Phasc D， Tab1c 1

YI Y2 Y3 Y4 SI S2 S3 b

。。 -0.25 -0.875 2.5 。 0.125 。ー0.1875 。 0.5 1.25 。 0.25 。 0.125 。 12.5

O 。 -3 5 6.5 。一4 w-31O

section. No proofs創'egiven. A¥l of the observations may not be relevant to Example 3.5 discussed above， but are incIuded here for completeness.

1. The primal and dual problems have the sam巴valuefor the optimal objective

function， 7." = w" = 280. This is true， however， only ifthe problems have optimal solutions.

2. (f x is any feasible solution to the primal problcm. and y is叩 yfeasible solulion to thc dual problcm.出叩 w(y)~ 7.(x). This feature provides an estimatc of the bounds of the dua1 optimal value if a fcasib1e primal solution is known. This rcsult also holds for the reverse case when a feasibl巴dualis known.

3. If出eprima1 problem is unbounded. the dual problem is infeasible. The

unbounded problem implies白紙theobjective function va1ue can be pushed to infinite limits.百lishappens if the feasible domain is not c1osed. Of coursc， practical considerations willlimit thc objective function value to corrcsponding Iimits on the design variables. The inverse relationship holds too. If the dua1 problem is unbounded， the primal is infeasible.

4. If the ith primal constraint is an equa1ity constraint， the ith dual variable is unreslricted in sign. The rcverse holds too. If the primal variable is unrcstricted in sign.出enthc dual constraint is an equality.

5. Obtaining primal solution from dual solution:

(i) If the ith dual constraint is a strict inequality， then出ei山primalvariable is nonbasic (for optimum solutions only). From the dual solution， the second

Toble 3.28 Example 3.5: Dual Problem， Phose 11， Tnble 2

YI Y2 Y.l Y4 SI S2 5J b

-2 。 -0.5 -0.5 。。2 。 0.25 。ー0.375 。 2.5 2 0.5 。 -0.25 。。15

8 。 -2 5 5 。。IV-300

:11

~i

3.5 ADDITIONAl TOPICS IN LlNEAR PROGRAMMING 147

Tab1e 3.29 Examp1e 3.5: Dua1 Prob1em， Phose 11， Tob1e 3

2 8

ー2

24

YI Y2 Y.l Y4 目.fl S2 SJ b

。。 -1.25 2 5 。。 -1.5 。 4 10 。 o 0.5 o -2 10

。。 5 2 。 8 .f -280

constraint (3.51b) is satisfied as an inequality. Therefo陀， the second primal

variable x2 is nonbasic剖 d出isis evident in Table 3.22.

(ii) If the ith dua1 variable is basic， then the ith primal constraint is a strict

equality. From Table 3.29， Y2 and Y3 are basic variables， therefore (3.47b) 叩 d(3.47c) must be equalities. This is indecd truc.

Thesc relations can also bc cstablishcd by valucs of slack variables.

6. Rccovering primal solution from final dual table: When lhe primal and dual problerns are in the standard form， the value of thc ith primnl variable equals the reduced coefficient (that is， the coefficient in the last row of the final tablc) of the slacklsurplus variable associated with the ith dual constraint. In Table 3.29， the values of the reduced cost coefficient corresponding to the surplus variables SI・S2'and S3 are 2， 0， and 8， respectively. Th回 eare precise¥y the optimal values of the primal variables.

7. lf出eith dual variable is nonbasic， the value of its reduced cost coefficient is the value of由巳 slacklsurplusvariable of the corresponding prima1 constraint. In Table 3.29， YI and Y4 are nonbasic variables wi山 reducedcost coefficients of 24 and 5， respωtively. These arc the values of SI (Slp) and S4 (S4p) in Table 3.22.

8. Obtaining duaI solution from primal solution: When the primal and dual problerns are in the standard form， the value of the ith dual variable equals the reduccd coefficient (白紙 is，the coefficient in the last row of the final table) of the slacklsurplus variable associated with出eith primal constraint. ln Table 3.22， the values of the reduced cost coefficient corresponding to the surplus variables SI， S2' S3， and S4副官 0，10， ¥0， and 0， respectively. These are precisely the optimal values of the dual variables in Table 3.29.

9. If the ith primal variable is nonbasic， the value of its reduced cost coeffici巴ntis

the value of the slacklsurplus variable of the corresponding dua1 constraint. In

Table 3.22， X2 is a nonbasic variable with reduccd cost coefficients of 5. This is

the value of S2 in Table 3.29. Since XI and X3 are basic， with the rcduced cost coefficient ofO，出ecor児 spondingslacklsurplus SI and S3 variables will b巴zero，as observed in T油le3.29.

The above list referred to the primal and dual problems in standard form. A similar list can bc obtained for nonstandard form. There will be appropriate modifications for


negative values of variablesぉ wellas equality constraints. The listed references can

be consulted for further informalion.

3.5.2 Sensltlvity Analysls

The solution 10 the LP problem is dependenl on the values for出ecoefficients， c， b， and A (also termedparameters)， involved in the problem. In many practical situations these par創nelersare only known approximately and may change from its current value for any number of reasons， particularly afler a solulion has been established. Instead of recomputing a new solution it is possible to obtain a fresh one based on the

existing one and its final Simplex table. This issue is significant in practical problems with thousands of variables and constraints. The adaptation of a n巴wsolution without re-solving the problem is called sensitivi.ηanalysis. In LP problems， sensitivity analysis refers to determining the range of parameters for which the optimal solution still has the same variables in th巴basis，even though values al Ihe solulion may change.

Example 3.1 is used for出isdiscussion. Figures are出 edfor i1Iustration rather than a formal proof. Figures are not useful for more than two variables. Generally， mosl compuler codes that solve Iinear programming problems also perform sensitivily analysis. Revisiting Example 3.1， it w悩 necessary10 identify the number of component plncemcnt machines of lype A and B. The objective is to maximize山e

number of boar由 tobe manufactured. Constraint g I represents the acquisition dollars available. Cons紅'aintg2 represents the floor space constraint. Constrainl g3 represents the number of operators available. The problem statement and the results are

reproduced below (note白紙X3'X4 and Xs are thc slack variables)

The solution is

Minimize j(X): -99OxI -9∞1X2 -5250

subjecI to: gl(X): O.4xl -+・ 0.6X2+ X3 = 8.5

(3.5)

(3.6)

g2(X): 3xI -X2 + X4 = 25 (3.7)

g3(X): 3xI + 6x2 + X5 = 70 (3.8)

x1 ~O; x2ミox3 ~ 0， x4 ~ 0， Xs ~ 0 (3.9)

x; = 10.4762， X; = 6.4285，x; = 0.4524，] = -21437.14

Changing Cost Coefficlent Values (c): First consider the e仔ectof changing

the cost coefficienls of the dcsign variablcs. If CIO the coefficicnt of design variable XI・is changed from -990， will the set of basis variables rcmain the same?

20

5

30 g s

舌ー1白

星-15

-20

-30 0

3，5 AODlTIONAL TOPICS IN LlNEAR PROGRAMMING 149

Sensltlvlty 10 Cost Coefflclent 25

電?・・・・・・・ーー司・.- 一一ー・・・・・・・ーーー-一一ー司・--ー---‘・・-句、司・・・ち片・・・‘ー16150

-・.句 --司、ー16150

-16150

c1 =ー200-optlmum changed 10 A

5 10 15

Machlnes of Type A

円guro3.9 Sensitivity analysis， changing cosl c佃町icient.

Changing thc cost coefficient changes the slope of the line representing出eobjective function血atmay lead to a change in批 solution.For example， Equation (3.5) can be writtenぉ

x2 =ー{990/900)X Iー([5250+ j]/900)

If CI were to be made -1900，出enthe slope of白eline will beー(1900/900).The sensitivity due to CI can be experienced by running Sensltlvlty_cost.m.1 The change wilh respect 10 the original objective function is shown in animation. The original solution is marked on the figure by a circle. The original solulion is unchanged when thema伊 itudeof the slope is raised， for instance CI = -1900. On the other hand， if CI were 10 be made -200， Ihe dotted line indicates the new cost line and in this case the solution wi1l change to point A. This is shown in Figure 3.9白紙 is山E問sultof running the m-fiIe. The floor constraint is no longer binding. It cun be established by simple ca¥culations白紙 infinitelymany solutions are possible if CI has a value of -450. In Ihis casc the cost function is parallel to the binding cons凶 int83' From this discussion it is apparent白紙 10keep the location of the original solution unchanged，

CI must be grealer than -450. Such anulysis is possible for all Ihe other cosl coefficienls involved in山isproblem. In particular， the analysis should indicate for

lFiles 10 bc downlonded from tl1e web silc nrc indicnled by boldface阻同時目ftype

150 L1NEAR PROGRAMMING

what range of cost coefficients the solution would still remain where it is. This is referred to as determining sensitivity to cost coefficients.

Change In the Resource Llmlts (b Vector): When the valu巴Son thc right-hand side changcs， the constraint Iineslplan岱/boun血riesare moved pamllcl to themselves. 前lischanges the feasible region. Th巴refore，the optimal vaIues may aIso ch叩 gedue to the changes in出efl回 sibl巴隠:gion.Once again the changes can be illus回 tedby running 血巴Sens耐vi町一巾s.m.Figure 3.10 iIIus回 testhe change in the right-hand side vaIue of 出efrrst∞nstr泊nt.b..針。mits vaIue of 8.5 to 6.5. This chang巴caus郎副 tobecome an active cons佐aint.In this case， the optimaI solution has moved from A to B. Note that in the originaI problem g 1 was not active. In血enew formulation it is. However， in sensitivity analysis，出eproblcm is to discover thc range of b. so白紙 thesolution still has由esamevariables in出ebasis. lf bl we陀 toremain above 8.0475， then the solution would still be atA卸 dgl will not be an active cons町aint

Change In the Coefflclent Matrlx A: A change in the coefficient matrix is similar in effect to that due to the change in the cost coefficient， while directly

20

15

10

5

白

星。ド

o -5 明也E

号・10是

-15

-30 0

Sensltlvlty to resource IImlts

b1:6.5-optimum changed-active constraint

5 ro 15

Machlnes 01 Type A

F1gure 3.10 Sensillvity analysis， changlng right・handside.

REFERENCES 151

20r Sensitivity to constraint coelficients (a11)

lIil~O

-‘ ‘・ -・.-‘ .... .. ・.

ーニご":----一『司司邑

5

@

ト~・。

ち」

-15

-20

m y a1':0.6 -optlmum changed

。 s 10 15

Machines of Type A

Flgure 3.11 Sensillvily analysis， changing cosl coefflclenl.

impacting由巳problcmby changing the feasible region. These changes also depend on whether the variable in the column is a bnsic varinblc 01' a nonbasic variublc.

Sensltlvlty_coeff.m illustrates the result of changing the coefficient all' An increase in the vaIue moves the optimaI solution to B， while a decrease leaves the

solution unchanged. In both situations， XI and x2 are still part of出ebasis. In the first case， g. becomes a binding constraint. This is illustrated in Figure 3.11. Thc coefficient is in the flf冨tcolumn， which corresponds to a basic variable. Changing coefficient values in the first column in other constraints wi1¥ yield similar information. The reader is encouraged to try other changes in the coefficient using出c

m-files available in the code section of this chapter.

REFERENCES

1. Dantzig， G. B.， Linear Programming and Extensio叫 PrincetonUniversity Press，

Princeton. NJ. 1963.

2. Luenberger， D. G.， Linear and Nonlinear ProgrammilJg・Addison-Wesley.Reading. MA.1984.


3. Winston， W. L.， lntroduction to Mathematical Programming. Applications and AIgorithms， Duxbury P陀ss，Bclmont， CA， 1995.

4. Arora， J. S.，lntroduction ωOptimal Desigll. McGraw-HiII， New York. 1989.

5. Wil1iams， G.. Linear AIgebra with Applications. Wm. C. Brown Publishers， Dubuque. IA， 1991.

6. Nob1c.B..飢 dDaniel， J. W.， Applied Linear AIgebra.針entiα-Hall，Eng1ewood Cliffs. NJ.1977.

PROBLEMS

(For all two-variable problems， provide a graphical definitionlsolution of thc problem.)

3.1 Solve the following Iinear programming problem:

Min ft..xl・X:z}:XI +X2

Sub: 3xI -X2 ~ 3

XI +2x2~5

XI+X2S4

XI ~O;X2~O

3.2 Solvc the following linear programming problem:

Max f(x)o xz): XI +巧

Sub: 3xI -xz~ 3

XI +2x2 S 5

XI+X2S4

XI 2! 0; X2 2! 0

3.3 Solveth巴followinglincarprogrammingproblcm:

Min f(x" X2): XI -X2

Sub: 3xI -X2 ~ 3

XI +2x2S5

XI +X2S4

XI2!0;X22!O

3.4 Solve the following Iinear programming problem:

M田 f(x"X2): XI + X2

Sub: 3xI -X2 2! 3

XI +2x2S5

XI+X2S4

XI2!0;X22!0

3.5 Solve出efollowing Iinear programming problcm:

'‘A':_ 1'1.. 、・ u ・..

Sub: 3xI -X2 S 3

XI +2x2~5

XI +x2~4

XI2!0;X2u町四回c旬din sign

PROBLEMS 153

3.6 The local bookstore must determine how many of each of the four new books on photonics it must order to satisfy the ncw interest generated in the discipline.

Book 1 costs $75， wiII provide a profit of $13， and 問 quires2 inches of shelf

spacc. Book 2 costs $85， wi1l provide a profit of $10， and requires 3 inches of shelf space. Book 3 co蜘 $65，will provide a profit of $8， and requir，ω1 inchof

shelf space. Book 4 costs $100， will provide a profit of $15 and requires 4 inches of shelf space. Find the number of each type that must be ordered to m蹴回ize

profit. Total shelf space is 100 inches. Total amount available for ordcring is

$5000. It has been decided to order at 1east a to凶 of¥0 of Book 2 and Book 4.

3.7 The local community college is p1anning to grow出巴 biotechno10gyoffering through new fcderal and state grants. An ambitious program is bcing p1anncd

for rccruiting at least 2∞stude附 fromin and out of state. They are to recruit at least 40 out-of-statc students. They wi11 attempt to陀氾ruitat least 30 students who are in the top 20% of their graduating high school c1ass. Current figu問Sindicate白紙about8% of the applicants from in state， and 6% of the applic叩 ts

from out of state belong to白ispooL They also plan to recruit at least 40 students who have APcourses in bio10gy. The data suggest that 10% and 15% ofin-state

and out-of-stale applicants， respective1y， bclong to出ispool. They anticipalc

that the additional cost per student is $800 for cach in-statc sludent and $1200 for each out-of-statc student Find thcir actual enrollment necded to minimize

cost and their actual cost.

3.8 Figure3.12rep陀sentsan optimization problem to delermine the max.imum tOla1 number of出巴 smaI1errectang1es山atwil¥ fit within thc larger one. The dimensions are indicated on白efig山"C.τnenumber of the 1arger r芭ctang1eshould be at 1east 5 morc than the sma¥1er oncs. Thc rectangles cannot be rolated. No fractional r巴clanglesare al1owed.

(Optional: confirm aIl solutions using the Optimizalion Toolbox. from MATI.AB.)

100

t仁コ1.5

2.5口 70

Rguro 3.12 Probl日m3.8.

4

NONLINEAR PROGRAMMING

Optimization problems whose mathematical models are characterized by nonlinear

equations are called Nonlinear Programming 例LP)problems. In Chapt巴r3 it wぉ

noted血atthese problems also fell into the category of mathematical programming problems. Engineering design problems are mostly nonlinear. In Chap旬r2 several

problems were examined graphically nnd it was evidentthat curvature and the gradient

of the functions involved had a significant influence on the solution. In subsequent

chapters we will continue to center由ediscussion of optimality conditions and

numerical tcchniques around two-variable problems because出eideas can also be

expressed graphically. Extension to more than two variables is quite straightforward

and is most simple when the p問 sentationis made through the use of vector a1gebra.

MATI.AB will be utilized for all graphic needs.

Traditionally， there is a bottom-up presentation of material for nonlinear

optimization. Unconstrained problems are discussed first follow巴dby constrained problems. For constrained problems山eequality constrained problem is discussed first. A s回ilarprogression is observed with問 gardto the number of variables. A

single-variable problem is introduced， followed by two variables which is由en

extended to a genernl problem involving " varinbles. This order allows incremental

introduction of new concepts， but primarily allows the creative use of existing rules to establish solutions to the extended problerns.

An analytical foundation is essential to understand and establish the conditions山at

the optimal solution will have to satisfy.百lisis not for the sake of mathematical

curiosity， but is叩 essentinlcomponent of the numerical technique: notably the

stopping criterin. The necessary mnthemntical definitions and illustrations are

introduced in this chapter. References ar巴nvailnblefor refreshing the calculus and the

154


numerical techniques essential 10 the development ofNLP [1，2). The books familiar to the render should do admirably. This chnpter also introduces the symbolic compuωtioll (computer algebra) resource availnble in MATLAB， namely， Symbolic Math Toolbox [3).

4.1 PROBLEM DEFINmON

In NLP it is not essential山atall the functions involved be nonlincar. It is sufficient if

just onc of them is nonlinear. Ther芭 aremany examples in engineering in which only

白eobjective function is nonlinear whi¥e the constraints are linear. lf in this case the

objective function is a quadratic function， these problems are termed lillear quadralic problems (LQP). Optimization problems for恥 mostp制 relyon experience to

identify the mathematical model comprising of the design variables， objective， and the constrainls. A knowledge of engineering， or the appropriate discipline， is also

essentialto cstablish a matllematical model. Primarily， this involves determining the functional relationships among thc design variables. Thc remaining task then is 10

establish the solution. How does one establish the solution to the nonlinear optimization problem? In mathematics (nfter nll at this stagc Lhere is a mathematical model for the

problem) the solutioll is obtained by satisfying the necessaηand SI{伊cientconditions related to the class of proble~. The necessary conditions ar官 thoserelations that a

C佃 didatefor the optimum solution mllst sati.φ.Ifit does，出en，and this is important，

it may be an optimal solution. To qualify a design vector XP (X represents the design vector) as an optimum， it must sntisfy additionnl relations cnlled the sujJicient conditions. Therefore，佃 optimumsolution must satisfy bo出 necessaryand sufficient

conditions. This chapter establishes these conditions for the optimization problem.

Example 4.1 is established next and is used in several ways in the remainder of出E

chapter 10 devclop the conditions mentioned above. Once 1hωe condilions are

available， the numerical techniques in optimization wiU incorporate them to establish

the solution.

4.1.1 Problem Formulatlon-Example 4.1

The problem is問strictedto two variables to draw graphical support for some of白E

discussions. There are two constraints， which， during the development of this chapter，

may switch betwccn equality and inequality constraints to i¥lustrate問 latedfeatures.

Problem: Find thc rectangle of thc largest ar回 (in血巴positivequadrant)白atcanbe

町anscribedw地ina given ellipse and satisfy a prescribed linear constraint. From出cproblem specification， the ellipse will provide an incquality constraint

while山elinear relation among thc variablcs will provide加 cqualityconstraint for

tbis cxample.

156 NONLlNEAR PROGRAMMING

3

2.5

x z

0.5

//

/

/

/f

Jff

//

。。 0.5 1.5 2 2.5

X1

Flgure 4.1 Conslrainls of Example 4.1.

Mathematlcal Model: Figure 4.1 capturcs the es総帥ofthe p帥 lcm.Code filcs

Ex4L1.ml and create_elllpse.m are filcs n田 cssaryfor MATI.AB to create the

figure.τ'here are two variables X I and X2・前lereare standard mathematical expressions

for山eellipse， and the straight line is no problcm after Chapter 3.

Standard Format: The standard format of the NLP is reproduced here for convemence:

Minirnize f(xtt X2' • • • ， Xn) (4.1)

Subject ω hk(x.，h・・・.，x，，) = 0， k = 1，2ド.，L (4.2)

め(XI，X2・・.. ， Xn) $; 0， j = 1， 2， . • . ， m (4.3)

xl $; Xj $; xl' ;= 1，2，... ，n (4.4)

In vector notation

IFiI田 10be downloaded from Ihe web silc町cindicllled by boldfnce悶悶悶riflype.

3

臨玄オ 4.1 PROBLEM DEFINITION 157

Minirnize f{X)，医ln (4.5)

(4.6) Subject to: [h(X)]， = 0

[g(X)]m $; 0 (4.7)

X10w $; X $; Xup (4.8)

For the specific problem being discussed， and refening to Figure 4.1， the design variables are the cool'dinate values xl叩 dX2山alallow thc computation of the

rect如忠lIararea. The optirnization problem is

Minirnize j{XI ， X~: -XI X2 (4.9)

(4.10) Subjectto: hl(x.， xz}: 20xI + 15 X2 -30 = 0

81(X.，X2): (xT/4) + (~)一 1$;0 (4.11)

O$;xl$;3; O$;x2$;3 (4.12)

The side constraints in (4.12) can also be postulated as one-sided and can be written

as

X1 ~O; x2~0 (4.13)

4.1.2 Dlscussion of Constraints

Using the relations (4.9)ー(4.12)several additional c1asscs of problems can be described by including only a subset of thc re1ations. They are examined below.

Unconstralned Problem: Thcre are no functional constraints although the side constraints arc necessary to keep the solution finite. For this cxample，

Minirnize (4.9) j{XI， ~): -x1 ~

In this problem， if出巴 designvariablcs are unbounded at山cupper limit， then血e

solution would be at the largest positive valu巴ofXI and X2・Atwo-sided limit for the

dcsign variables is usually a good idea. The designer does have the回目ponsibilityof

defining an acceptable design space.

Equality Constralned Problem 1: The functional constraints in this problem紅巳

only equalities. With reference 10 Example 4.1出cfollowing problem can be scl up

(after changing曲cinequality to an cquality):

158 NONL1NEAA PAOGAAMMING

Minimize ft..x.， X2): -x. x2

Subject to: 11)(x.. X2): 20 x. + 15 X2 -30 = 0

h2(x.・X2):(~/4) + (4)ー 1=0

O~XI ::;;3; 0~x2::;;3

Intuitively， such a problem may not b巴 optimizedsince the two constraints by

themselves should establish the values for the Iwo design variables. The argumenls used in LP for acceplable problem definilion are also valid h巴悶.There is always the

possibility of multiple solutions-which is a strong fealure of nonlin巴arproblems. In

such an event the set of variables that yield出elow巴slvalue for the objective will be the optimal solution. NOle白紙sucha solution is obtained by scanning the acceptable solutions rather白anthrough application of any rigorous conditions.

Equallty Constralned Problem 2: Ifthe problem we偲 10incIude only one of lhe constraints， for example，

Minimize ft..xl・X2):-XI X2

Subjecllo: h2(x"X2): (~/4) + (~)ー )=0

0::;; XI ::;; 3; 0::;; X2::;; 3

This is a valid optimization problem. A similar problem can be defined wilh the first constraint by itself.

Inequallty Constrained Problem: In this case the constraints arc all inequalities. A variation on Exarnple 4.1 would be (the equality cons町aintis transformed to an equality constraint)

Minimize ftx.. x2l: -XI Xz

Subjcct to: g.(x" X2): 20 XI + 15 X2 -30::;; 0

g2(X"XZ): (xt/4) + (4)ー1::;;0

O::;;XI ::;;3; 0::;;x2::;;3

Like its counte中町tin linear programming，出isis a valid oplimization problem.

Equally valid would be a problem that incIuded just one of lhe constraints or any numberofin叫ualityconstraints.

It is essenlial to understand both the nature and the number of constraints as

well as how they affect the problem. In general， equality constraints are easy 10 handlemathematICally，difficulttosatisfy numerically.andmorerestrictiveonthe search for the solution. Inequality constraints are difficult to resolve mathematically and are more flexible with respect to the search for the optimal solution as they define a larger feasible region. A well-posed problem requir~s that

4.2 MATHEMATICALCONCEPTS 159

someconstraintsareactive (equality) at the optimal solution as otherwise it would

be an unconstrained solution.

4.2 MATHEMATICAL CONCEPTS

Like LP， some mathematical definitions are n田凶sarybefore the n町民saryand

sufficient conditions for the NLP can be established. Definitions are needed for both

白eanalytical discussion as well as numerical techniques. MATLAB provides a Symbolic Math Toolbox which permits symbolic computation integraled in出c

numerical environment of MATLAB. This allows Ihe user 10 explo陀 problemsin

calculus， linear algebra， solutions of system of equations， and other areas. In fact， using symbolic computation students can easily recover山epr邸宅quisiteinformation

for the course. A short hands-on exercise to symbolic computation is provided.1τ'he primary definitions we need are derivatives， partial derivatives， matrices， derivatives of matrices， and solutions of nonlinear equations.

4.2.1 Symbollc Computatlon Uslng MATLA目

One of the besl ways to gel fami1iar with symbolic computation is to take the quick online introduction 10 the Symbolic Math Toolbox available in the MATLAB Demos

dialog box [4]. The following sequence locates the lutorial:

>> demos

Symbolic Math -・> Introduction

The computalional engine execuling由巳 symbolicoperations in MAT1.AB is thc kemel of Maple markeled and suppo巾 dby Waterloo Maple， Inc. lf出E民 aderis

already familiar with Maple，出enMAT1.AB provides a hook through which Maple

commands can be ex田 utedin MATLAB. The symbolic computation in MATLAB is performed using a symbolic object or sym. This is another data type Iike the number and string data types used in阻 rlierexercises. The Symbolic Math Toolbox uses sym objects to陀 presentsymbolic variables， expressions， and matrices.

In the exercise that follows， a function of one variable， and two functions of Iwo variables (constraints from Example 4.1) at1巴 usedfor iIIustralion. Drawing on the au出or's c1assroom ex戸巾nce血ispreliminary discussion is in terms of variables X

and y for improved comprehension. In later sections， subscripts on x are used to define multiple variables so that出etransition to the general problem can be facilitated using

vector description. The functions in thcsc exeaises are

ftx)= 12+(x-I)2(x-2)(x-3)

g.(x，y): 20x+ 15 y -30

g2(X，y): x2/4 + l-1

ITOO rclcasc of 100 new Swdent Edilloll includcs Ihc symbolic loolbOl¥ ulong wilh Ihe basic MATLAD sortw町t

161 4.2 MATHEMATICAl CONCEPTS

、，Y

，

噌・-g

'E=

SE. f

S

5

・1

n

l

d

a

者 9column vector based on gl，g2 9 '" [gl;g2)

9 [ 20*x+15*y-30]

1/4*x+y-1]

も 9 can be the constraint vector in optimization 者 problems

も thepartial derivatives of 9 with respect to design 苦 variablesis called the Jacobian matrix も theproperties of this matrix are important for 者 numericaltechniques

xy .. [x y]; も rowvector of variables J '" jacobian(g，xy) も calculatingthe Jacobian

15] 1]

者 aplot of f for -2 pi

df 司 diff(f);h01d 00

ezp10t(df， [0，41) 告 plottingfunction and derivative 者 combinewith MATLAB graphics -draw a line 1ioe ([0 4]， [0 0)，' C010r' ，， r' ) g

9 [ 20*x+15*y・30]

1/4*x+y-1]

(default) 2 pi

<= 4

J

20， [ 1/4，

ezp10t(f) x

者 plotbetween 0 <= x ezp10t(f， [0，4))

"J'ljl

NONLlNEAR PROGRAMMING

百lefollowing MATLAB session was capωred as a diary file and edited in a text editor. The児 fo陀山eMATLAB prompt does not appe紅.百leboldface words are commands出at山ereader will type at the command line.

160

者 definingx as a single symbolic object x "， sym('x') x x

者 definitionof multiple objects variables in the workspace

Bytes Class 126 sym object 126 sym object 128 sym object 128 sym obje'ct 126 sym object 126 s戸nobject

g

of g1 g2 types Size 1X1

1x1 1X1

1x1 1X1 1X1

syms y f whos % Name f

g gl

92

x y

Grand total is 14 elements using 760 bytes

者 constructingf (x・1)*(x-1)*(x・2)*(x・3)f.. 12 +

f

12+(x-1)"2*(x・2)*(x-3)

diff(ま}aos 2*(x-1)*(x-2)*(x-3)+(x-1)2*(x-3)+(x-1)"2*(x-2)

も firstderivative

Y 2.5

Additional symbo1ic computations will be introduced through code as appropriate. Note that白eresult of both numeric and symbolic computations can be easily combined a10ng with graphics to provide a powerful computing environment.

も toevaluate 9 at x 1， subs(g，{x，y}，{1，2.5}) ans

27.5000 1.7500

3 5 書

室読

者.ZO:~，

選

定

3 2究

も notethe chain rule for derivatives % note the independent variable is assumed to be x

diff(ま，x，2)

ans 2*(x-2)*(x・3)+4*(x-1)*(x-3)+4*(x-1)*(x・2)+2*(x-1)"'2

者 thesecond derivative wrt x

主 thethird derivative wrt x }

弓

a，

X

2

'nHE

aL=

・

(

x

f

g

*

f

n

4

1

a

2

d

告 define9工nu

qd y

-wnuw

Ed

司

3

唱

・

-

-

A

T

Y

*

X

5

*

句

4

0

+

2

x

=

*

E

n

u

噌

t-14内，

a

g

g

も defineg2

gl，g2 can only have partial derivatives independent variables have to be identified

}

x

，

t-g

't=

8

・-aιEMnu

i

n

2

d

a

も partialderivative

g2 '" 0.2S*x + Y ・1;

者

も

162 NONLlNEAA PAOGAAMMING

4.2.2 Basic Mathematical Concepts

The basic mathematical e1ements in the discussion of NLP are derivatives， parti叫derivatives， vectors， matrices， Jacobian， and Hessian. We havc used the Symbolic Math Toolbox in the previous scction to calculate some of these quantities without dcfining them. These topics will have been extensively covered in the foundation courses on mathematics in most disciplines. A brief review is offered in血issection using MATLAB.百lisopportunity will also be utilized to increase familiarity wi出theSymbolic Math Toolbox and incorporat巴 itwith出eMATLAB commands that were used in山巴earlierchapter冨.

Functlon o'One Variable: ft..x) identifies a function of one variable:

f(x) = 12 + (x-li (x-2) (x-3)

is used as a specific example of such a function. The derivalive of出efunction at山elocation x is written as

玉江-1・kx + ax) -.f(x) γ 笠dx-ilm Ax =llfI1-L

ax→o ax ax->oax

(4.14)

x is the point about which由巴 derivativeis compuωd.ムris thc distamニetu a neighboring point whos巴locationtherefore will be x + ax. The value of the deri vati ve is obtained as a limit of thc ratio of白edifferen田 inthe value of the function at曲etwo points (幼 ω出edis岡田 separatingthe two points (位)，ぉ thissep叩 tionis reducedωzero. The computation of thc derivative for the specific ex創npleis usually obtain氾dusing the product rule. Results from the exe民iseon symbolic computation provide thc resuJt

ま=均一例一例ー 3)+ (x -1)ゆい)+ ト 1) 川ー2)

Table 4.1 illustrates出elimiting process for thc derivative of the function chosen as 白ecxampJe. The derivative is being evaluated at point 3. As the dispJacements are reduced，出巴 valueof the ratio approachcs出巴 valueof 4 which is the aClual value of the derivative. When ax is large (with a value of 1)，出cratio is 18. This is significantly different from the value of 4. With ax of 0.1 the value is around 4.851. With fur也erreduction to O.∞1 the ratio has a value of 4.008. From these computalions it Can bc expected白紙 asax approaches 0 the ratio will reach the exact value of 4.0.

Numerical Derivative Compu削'ion:Many numcrical techniqucs in NLP (1叫ui陀

出ecomputation of derivatives. Most software implementing these techniques do not use symbolic computation. Automation and回 seofuse毘quire血at也esederivatives be computed numerically.百leresults in Table 4.1 justify出enumerical computation of a derivative through a technique called thefirslforward d，砂量rence[5]. Using a very small pe目urbationax， lh号derivativeat a poinl is numerically calculated as the ratio

4.2 MATHEMATICAL CONCEPTS 163

Table 4.1 Calculation or DerivaUve

x dx x+dx M dfldx Derivative

3 4 18 18 4 3 0.1 3.1 0.4851 4.851 4 3 0.01 3.01 0.0408 4.0805 4 3 0.001 3.001 O.∞4 4.008 4

of the change in the function value， 4f， 10出echange in恥 displacemenl，ax. For出E

exampJe， the derivalive at x = 3 with ax = 0.001 is obtained as

ElI _f.:(3 + 0.001)ー1(32-.11nnSl

dxl.rc3 0.001 ー

(4.15)

The derivativc for lhe singJe-variable function al any valuc x is aJso caJled lhe slope or the gradietlt of lhc function 創出alpoint. If a line is drawn tangenl 10 the function al the value x， the tangent of白E叩 glethat出isJine makes wi出thex-axis will have the s創nevaluc as the derivative. If血isangle is 9，出en

ま=tan9

Figure 4.2 illustrates出etangency property of the derivative.

MATLAB Code: Figure 4.2 was crealed by the code tangent.m. In the same directory，出ecode derlvative.m provides the illustration of tangency through a fi伊 reanimation in MATLAB. In出efigure window both the function (drawn as the curve) as well as the line representing 4fand ax are drawn. As the neighboring points arc closer， it is evide似合omthe figure that the curve can be approximated by a straight line. When ax = 0.001，出ederivative is tangent to the curve as出eyare indistinguishable.

H匂herDerivatives: The derivative of the dcrivative is the called出esecond

derivalive. Formally it is dcfined邸

互主-!!..(笠-li"，坐笠生L!;rn炉心1.. 出 -4yalxtJil-dx ldx)一心--10 ax 似:ゐ ax (4.16)

Similarly the lhird derivalive is defined as


24

22

201-・"

18

12

10

a o 0.5

tangent -slope -derlvative at x=3

lo12+(x・t)・(x-l)・(x-21・(x-3)

1.5 2 x

2.5

Flgure 4.2 lIIuslrallon 01 the langent.

表=増)

3 3.5 4

(4.17)

This can go on provided出efunction has sufficient higher-order dependence on出E

independent variable x.

Function of Two VBriBbles: The lwo-variable function

j{x川:x214+y2-1 (4.18)

is chosen to iIlustrate the required mathcmatical concepts. The first important fcature

of two or more variables is白紙 thederivatives defined for a single variable (ordinary

derivatives) do notapply.百】eequivalent concept here is the partial derivative. Partial derivatives are dcfmed forcach independcnt variable.百lepartial derivative is denoted

by the symbol d. These dcrivatives are obtained in the same way as the ordinary derivative except thc other variables are held at a constant value. In the cx創nple，when compuling白epぽtialderivative of x thc value of y is not allowed to change.


dfl _I;~~(x + ax， y) -f(x， y) (4.19)

dxl 二ム /:u 1(.<.)1)山山

Thc above relation expresses the partial derivative of the functionfwith respect to出E

variable x. Graphically this suggcsts that thc two points are displaced horizontally (assuming the x-axis is borizonta1). In the above expression (and prior ones too)， the subscript after the verticalline establisbes tbe point a加utwhich出epartial derivative with 郎 pectto x is being evaluatcd. (x，y)即応se附 any/allpoints. A similar expression c四 be

written for the par首alderivative offwith陀spectto y. For this example:

For the pointx = 2， y = 1:

ー[-~.司f-一一:"=2ydx -2' dy

df • df f(2，I) = 1; 一 =1・ー=2

dx-" dy

Until this point wc have made use of symbols reprcsenting changes in values (functions， variables) without a forrnal definition. In山isbook

s():犯 presentsfinitelsignificant changes in the quantity ( )

d( )， o( ) : represents diぽercntial/infinitesimalchanges in ( )

Changes in functions occur due to changes in the variables. From calculus [1] thc di仇 rentialchange inj{x，y) (ゆ dueto the differential change in血 variablesx (dx) and y (dy) is expressed as

L" dj，. df df= -:J dx + -:J dv "J dx _. . dy-J (4.20)

For convcnience and simplicity， the subscript representing白epoint whcre the exprcssion is evaluated is not indicated. The definition of the partial derivative is apparent in the above expression as holding y at a constant value implies出atの=0.

Another interpretation for the partial derivative can be observed from thc above definition: cltange of tlte function per unit citange in tlte variables.

Gradient 01 the Function: In the function of a single variable， the derivalivc was associated wi出 theslop古.In two or more variables the slope is equivalent 10山e

gradient. The gradient is a vector， and at any point rep児 sentsthe direction in which the function will increase most rapid，かExarniningthe conventional objective of NLP， minimization of objectiveβmctions， the gradient has a natural part to play in the

development of methods to solve the problem. The gradient is composed of the partial dcrivatives organized as a vcclor. Vectors in白isbook are column vectors unless othcr

noted. The gradient has a standard mathematical symbol. lt is dcfined as


Vf日=阿 (4.21)

At this stage it is appropriate to consolidate白isinformation using graphics. The

graphicaI d巴scriptionof白eex創npledefined in Equation (4.18) has to be three dimensional as we need one axis for x， ano出erfor y， and the third forj{x，y). Chapter 2 introduced three-dimensional plotting using MATLAB. A more useful description of

the probl巴mis to use contour plots. Contour plots are drawn for specific values of the function. In the iIIustration the values for the contours offare 0， 1，2，叩d3. In Figure

4.3， two kinds of contour plots are shown. In the top half a three-dimensional (3D)

contour plot is shown， while on白 lowerhalf恥 sameinformation is presen凶ぉa

two-dimensional (2D) contour plot (this is conside陀 dth巴standardcontour plot). Thc

2D contour plot is more useful for graphicaI support of some of the ideas in NLP. The

3D contour will be dispens巴dwith in山eremainder of thc book.

MATLAB Code: Flg4_3.m. The annotation in Figure 4.3 (including the tangent linc and the gradient町 ow)wasdone出roughthe plot吋itingcommands available on山emenu bsr in the figure window (MATLAD version 5.2 and later). A m吋01"pOl"tion uf 白cplots a問 generatedthrough the statcments in the m-fiIe indicated above. The codc

mixes numeric and symbolic computation. It allows multiple plots and targets

graphic commands to specific plots.

Discussion o( Figure 4.3: In Figure 4.3 point P is on contour f = O. Point Q is on

the contourf= 2. Point S is on the contourf= 1. Point R has出esame y valuc as point P佃 d白esame x vaJuc as point Q. For出evalues in the figure the contour value is 0.75

(this value should be displayed in MATLAB window whcn the cod巴Flg4_3.mis run).

line PQ is a measure 4f

line PR represents changes in dfwhcn d.y is 0

lineRQ問presentschanges in dfwhen d.x is 0

Mathematically，ムfcan only be estimated by adding the changcs along the lines PR andRQsince出cdefinition of the partial derivatives only pcrmits calculating changcs

along the coordinate directions.

At point S the dotted line is the tangent. The gradient at thc samc point is normal (pcrpcndicular) to山ctI.ngent岨 dis dir田 tedtoward increasing the vaJue of白efunction (indicated by白earrow). By definition， if dfrepresents a differentiaJ move aJong the gradient at any point，出en(dx，めP副官 measuredalong the gradient vector)

df=空白+さdy(4.22)

3

2

0 3


3D conlour for f(川 1=x2/4+I-l

一一.

2 2.5 3

。。x

2Dconlour 3

2.5・ gllldlenl 01 S

-・， longenl 81 5

2 3 x

Flgur，由 4.3 Gradienl and langen! IIn自 a!a point.

From恥 figurethe value offwiIl change along gradient direction. If dfrepresents a differentiaJ move aJong恥畑gentline，白en(dX，のaremeasured aJong刷 gentline)

df= ~dX+主の=0批eJy

(4.23)

Equation (4.23) should bc zero because moving tangcntially (a small amount) the vaJu巴offisnot changed.

~acobian: The Jacobian [J] defines a useful way to organize the gradients of sevcraJ functions. Using threc variables and two func!ionsj{x，y，z) and g(x，y，z) the definition of thc Jacobian is

I eJf互主|

[JJ =1 ~x ~ ~~ 1 I~星笠笠|

(4.24)


In Equation (4.24) the gradients ofthe function app町 in出esame row. The first row

is出egradient of f while the second row is the gradient of g. lf the two functions are coUected into a column vector. the differential changes [df dg]T in山efunctions， due to出edifferential change in thc variables [dx dy dz]， can be expressed as a matrix

multipucation using the Jacobian

川州制

d

rill--L

]

官

4[

=

、lEE

・E

・-J4・勾

rBEE-EEEL

(4.25)

which is similar to Equation (4.20).

Hessian: The Hessian matrix [H] is the same as the matrix of second derivatives of

a function of several variables. For ft...x，y)

I dγaγ|

[問=1ぜぬ|I iJL[ iJL[ I I dydx ザ |

(4.26)

The Hessi叩 matrixis symm凶 ic.For出巴exampledefined by Equ瓜ion(4.18).

nv司&

ヲh，f

AU

問

Functlon of n Varlables: For ft...X). where匹]=[XI・X2.. . . • x，.rr.白egradient is

The Hessian is

笠 d'i ••• d立一à~ dx1iJx2 iJx1axn

~ a'i 問 =1dX2ax1 à~

~ aγ iJxndx， d..c

(4.27)

(4.28)

4.2 MATHEMATICAl CONCEPTS 169

Equations (4.27) and (4.28) will app回 rquite often in succeeding chapters. A few

minutes of familiarization will provide sustained compreh巴nsionlater.

4.2.3 Taylor's Theorern/Serles

Slngle Varlable: The Taylor series is a useful mechanism to approximate出evalue

of the functionft...x) at山 point(xp +お)if白efunction is completely known at point xp• 百le exp叩 sionis (for finitc n)

手I~ (制2+ ト古家|う州的+釘)叫，)+まI(お)サI~

The series is widely used in most disciplines to establish continuous models. It is出E

mainstay of many numerical techniques， incIuding those in optimization. Equation (4.29) is usually truncated ωthe first two or山町 termswi白血eunderstanding批

approximation will suffer some error whose order depends on恥 term伽 tis being truncated:

+ 玉江I{AY¥.1.1. fll [(Xp+At)三f(XJ&|Jh)+21dlJ釘 )2+O(位 )3(4.30)

lf the first term is brought to山eleft，出eequation. discarding the error term， can be written as

M何=イ吠f六(うxp+釘削)一f川(令句x巧ろ引:X/，)=戸，，)片)片=判(仰刷釘削)+上立|刷dxL目 21 dX-1 -Ab-xh

In Equation (4.31)， the first term on白eright is called thefirst-order/linear variation while the second term is出esecond-order/quadratic variation.

Figure 4.4 demonstrates the approximations using Taylor' s series of various orders at point 2.5 with respect to白巴originalfunction (red). The principal idea is to deal wi出the approximating curve which has known prope凶esinstead of the original curve.

百lCconstant Taylor series is woefully inadequate.百lelinear expansion is only marginally better. The quadratic expansion approximates the right side of the function comp釘吋 tothe left side which is in significant eπor.百lefifth・orderexpansion is

definitely acceptable in the range shown. To see how the Taylor series is used，

consider the quadratic curve about出cpoint x = 2.5. At this point出巴 valueof the

functionft...2.5) = 11.4375， the value of th巴firstderivativef'(2.5) = -0.75，皿dthcvaluc

of the second dcrivative isf"(2.5) = 4. Using Equation (4.31)

(4.31)

ft...2.5 +制 =1間 5+(-0万)釘+主榊(4.32)

For different values of At. bo山positivcand ncgative， the value ofj(x) can be obtained. 百leplot of Equation (4.32) should be the same as恥 onelabeled quadratic in Figure 4.4.

171

(4.34)

4.3 GRAPHICAL SOLUTIONS

f(xp+dx，ゐ+勾)=f(xp'ゐ)+町川引+、〆 I~YI

iMy蜘P，yp)][~]

(4.35)

Fornv副 ables，withろthecurrent point and d.X the displacement vector，

fぽp+ IlX) = f(Xp) +町鳥川+jmTm

GRAPHICAL SOLUTIONS

百legraphica1 solutions are presented for three kinds of problems: uncons位ained，equality constrained， and inequality cons凶 ined，based on the functions in Example 4.1.

4.3.1

o ~xl ~ 3; 0 ~x2 ~ 3 (4.37)

The sid巴constraintss巴rveto limit the design space. Considering the objective function in (4.36)， it is clear that the minimum value of f wi1l be r巴alizedif the variables are at the maximum (Xi = 3，巧=3). The value of the function is -9. Figure 4.5 il1ustrates the problem. The solution is at the boundary of the design space. Fig4_5.m is the MATLAB code山atwill produce Figure 4.5. The maximum value of the function is 0 (based on the side constraints). In Figure 4.5 the tangent and the gradient to the objective function at the solution are drawn in using MATLAB plotedit commands. lD也isparticular example， the side constraints are necessary to determine the solution. If the design space were increased the solution would correspondingly change.

The巴xamplechosen to illustrate曲eunconstrained problem defined by (4.36) and (4.37) is not usually employed to develop the optimality conditions for general unconstrained problems. Two requirements are expected to be satistied by such problems: (1) the solution must be in the interior of the design space and (2) there must b巴 aunique solution， or白eproblem is unimodal. Nevertheless， in practical applications these conditions may not exist. Most software are developed to apply the optimality conditions and they rarely verify出eserequirements are being met. It is usua1ly the designer's responsibility to ensure these requirements are met. To develop 也eoptimality conditions an aItemate unconstrained problem is presented:

(4.36) JtXI，x2): -XI X2 Minimize

(4.38) JtXI，x2): (XI-1)2 + (X2 - 1)2 -XI X2 Minimize

，1t1 キ I

fifth-order a~d orlg!nal

NONLlNEAR PROGRAMMING

Approxlmat!on us!rtg Taylor serles

170

22

-

0

、a

Unconstralned Problem

4.3

20

宣

告18

i

E

12

4

MATLAB Code: Figure 4.4 is completely created using the code Flg4_ 4.m.1t uses the MATLAB-provided symbolic “tayloc" function to generate the various expansions about point 2.5.

3.5 3 2.5

Taylor series approximalion.

2 x

1.5

Figure4.4

0.5 D

Two or More Variables: The series蹴 onlyexpanded to臨 qu拙 'aticterms. The truncation error is ignored.τ'he two-variable function expansion is shown in detai1 and also organized in terms of vectors and matrices. The first -order expansion is expressed in terms of the gradient. The second-order expansion will be expressed in terms of tbe Hessian ma甘ix

まIMPI+f(x'J +似 Y/J+ Ily) = Jtx'J' y，) + 1さ|釘+Q，冗 1(.I"pろ}

手Ittp，Y/叶e)2f I 耳石(w-Y+

1 r笠| …2・q

21 ë)~I (x，川、ー〓・ー (4.33)

If批 displacementsare organized as a column vector [似勾]T，the expansion in (4.33) can be expressed in a condensed manner as

市72 NONLlNEAR PROGRAMMING

4

3.5

3

2.5

，.N 2

1.5

0.5

。。 0.5 1.5 2 x

' 2.5

Flgure 4.5 Unconslrained solution.

o $XI $ 3; O$X2 $ 3

3 3.5 4

(4.37)

Figure 4.6 displays the problem and its solution. Flg4_・.6.mprovides the code.that wiII generale most of Figure 4.6. It is cJear山atthe solution appears to be ot

xi = 2 and xi = 2. The optimal value of山eobjective function is -2. In the contour plot，出eoptimum is a point making it difficuJt to draw the gradient. If白isis the case，

也enin three dimcnsions， at the optimum，出cgradient wil¥ Iie in a plane tangent to the function surface. Moving in出isplane should not change the value of the objectivc function. This observation is uscd to develop the n田 essaryconditions later.

4.3.2 Equallty Constrained Problem

For a two-variablc problem we can only utilize one constraint for a meaningful optimization problem:

MinImize j{XI，x2):ー'XIX2

Subject ω hl(XI，x2): X!/4 + ~ -1

(4.36)

(4.39)

4.3 GRAPHICAL SOLUnONS 173

x t

円gure4.6 Unconstrained problem: interior回 1lIllon.

OSX1 $3; OSx2S3 (4.37)

Fi思u"e4.7 (由roughFIg4ー7.m)i1Iustrates the problem. The dashed line is白e

∞ns住aint.Since it is an equaJity constraint， the solution must be a point on the dashed

line. The contour of f =ー1appears to just gr回巴出econstraint and therefore is the minimum possible vaJue of the function without violating the constraint. In Figure 4.7，

血egradient of the objective function as weIl as曲egradient of白econstraints at the

solution ar官ilIustrated.It appears白atat血ispoint these gradienls are parallel cvcn

though出eyare directed opposite to cach other. By definition the gradient is in the

direction of thc most rapid increase of the function at the selected point. This is not a

coincidenc巴.This fact is used to establish the necessary conditions for the problem.

4.3.3 Inequallty Const悶 InedProblem

The number of in巴qualityconstraints is not dependent on the number of variables. For

i1Ius岡山n，bo白 cons佐直intfunctions of Example 4.1 ar芭 formulatedas inequality

cons町amts.

Minimize j{x)， X2): -XI X2 (4.36)


4

3.5

3

対刷 2

1.5

0.5

0.5 1.5 2 X 1

2.5

Flgure 4.7 Equality∞nstralned problem.

Subjectto: gl(XI，x2): 20xI + 15x2-30S0

g2(Xl> X2): xI/4 +えー ISO

OSXI S3; OSX2S3

3 3.5 4

(4.40)

(4.41)

(4.37)

Figure4.8 ilIustrates thegraphicalsolution. Thesolution isat the intersection ofthe dotted lines. The code is a vailable in Flg4_8.m.百legradients are drawn using the plotedit functions on the menubar in the window. Also font style and font size have becn adjusted using the plotedit commands. The optimal solution must lie on or to山E

left of the dashed line. Similarly it must Iie below or on the dashed curve. Simultaneously it should a1so decrease the objective function value as much as possible.

4.3.4 Equality and lnequallty Constralnts

Example 4.1 is developed with both types on constraints present. Il is問producedhere:

4

3.5

3

2.5

1.5

0.5

0.5

4.4 ANAlYTICAlCONDITIONS 175

1.5 2

x t

2.5

Flgure 4.8 Inequality constralned problem.

Minirnize j{X¥，x2): -xl x2

Subject 10: hl(XI，x2): 20 XI + 15 X2 -30 = 0

gl(XI，x沙:(xI/4) + (~)一 IS0

o S XI S 3; 0 S X2 S 3

3

(4.9)

(4.10)

(4.11)

(4.12)

4

The graphical solution of Section 4.3.3 suggests that the inequality constraint will be active at the solution. If白紙isthe case， then Figure 4.8 will once again represent出egraphical solution to this section凶so.This will be vcrified in thc ncxt scclIon.

4.4 ANAL YTICAL CONDITIONS

Analytical conditions rcfer to恥 n回 essaryand sufficient conditions伽 twill permit the recognition of the solution to the optimal design problcm. Thc conditions developed hcre empower the numerical techniques to follow later. They are


introduced in由巳 samesequence as in山eprevious section. lnstead of fonnal

mathematical details， the conditions are established less fonnally from the

geometrical description of由。 problemandlor出roughinωitive reasoning. Fonnal

dcvelopment of thc analytical conditions can be found in Refercnces 6-8. For

establishing the conditions it is expected that the solution is in the interior of山e

feasibl巴region，and there is only one minimum.

4.4.1 Unconstrained Problem

The problem used for illustration is

Minimize J(x.. X2): (XI -1)2 + (X2 -1)2 -XI X2 (4.38)

(4.37) OSxIS3; OSX2S3

Figurc 4.6 provided a contour plot of the problem. Figure 4.9 provides a

three-dimensional plot of the same problem. It is a mesh plot. The commands to create Figure 4.9 are available in Flg4_9.m. The figure needs to be rotated intemctivcly to

appear as illus回 ted.A tangent plane is dmwn 鉱山巴 minimumfor emphasis. Figure

4.9 wil1 be us巴dto identify the prope凶esofthe functionJ(xl，xz) at the minimum. The minimum is identitied by a supe凶criptasterisk (X・orlxi， X2JT). Studying

Figure 4.9， the function has a minimum value of -2， while xj = 2 and x2 = 2. If白e

values of X1 andlor X2 we民 tochange even by a slight amount from自己optimal value，

in any direction， the value ofthe function wil1 certainly increase since X' is the lowest point of the concave surface rep閃 senting出efunction f Reprcsenting出E

displacemcnt from the optimum values of the variables as dX， and thc changc in the

function valuc from the optimum as llf， from direct observation it is clear that the

optimal solution must be a point由atsatisfies

4f> 0， for al1 dX

Flrst-Order Condltlons: The same idca can bc applied in the limit， that is， for infinitesimal displacement dx1 and dx2 about X・.The function itself can be

approximated by a plane tangent to the function at the solution (shown in Fi酔re4.9)

(considcr血isぉ afirst-ord巴rTaylor series expansion). Moving to any point in the

plane from the optimum (see Figure 4.9) will not change the valuc of the function，

白E閃 foredf = O. Moving away from optimum implies白紙 dx1and dx2 are not zero.

Rewriting Equation (4.20) in tcnns of X1 and X2

df=か1+か=0

df=医者][:J=O (4.42)

4.4 ANAl YTICAL CONDIT10NS 1 n

10

4

B

6

組句、H

・FHX

2

。

-2 4

X 2

。。

F1gure 4.9 Three-dimensional plot 01 Figure 4.6.

Since this should hold for al1 points in白eplane， dx1 1:-0 and dx2 1:-O. Therefore，

df -fl 主 -fldX

1 -v， dX2 -v

or the gradient of f at the optimum must be zero.百 atis，

Vf(x~，x;) = 0 (4.43)

Equation (4.43) expresses the necessary condition， or first・orderconditions (FOC)， for unconstrained optimization. It is termed the frrst・ordercondition because Equation (4.43) uscs出egradient or the first derivative. Equation (4.43) is used to identify thc

possible solutions to the optimization problem. If出巴functionwere to flip over so出at

the same function wc陀 tomaximize the value of出efunction， the solution for the variabl巴swill be at the same value of the design variablc. It is clear that Equation (4.43) appliesωthe maximum problem also. Equation (4.43) by itself will not

determine the minimum value of the function. Additional considerations a偲 necessary

to ensure山atthe solution established by出巴 first-orderconditions is optimal， in出is

casc a minimum. For a general unconstrained problcm， the necessary conditions of Equation (4.43) can be stated as


Vf(X'・)=0 (4.44)

which is primarily a vector expression of the relation in Equation (4.43). Equation (4.44) is used to establish the value of the design variables X' both analytically and numerically.

Second-Drder Conditions: The second-order conditions (SOC) are usual1y regarded as sufficient conditions. It can be infe町'ed出atthese conditions will involve

second derivatives of th巴 function.The SOC is often obtained through the Taylor expansion of the function to sccond order. If X申 is出巴solution，and d.X represents白e

changeof出巴 variablcsfrom出巴optimalvalue which will yield a change dif， then

年刊念+6X)ーが)=可(X市川+t ah r凶 (4.45)

This is sirnilar to Equation (4.35) except出eexpansion is about出csolution. 6f must be grcater than zero. Employing the necessary conditions (4.44)， the first term on the right-hand side of Equation 4.45 is zero. This Icavcs出efollowing inequality

af=jaTH (4.46)

whereH(X・)is the Hessian matrix (the ma紅ixof second derivatives) of the function

f at thc possibl巴optimumvalue XへFor出erelations in Equation (4.46) to hold， the matrix H(X勺mustbe positive definite. There are three ways to巴stablishthe H is positive definite.

(i) For all possible .6X， d.XTH(X').6X > O.

(ii) The eigenvalues of H(X') are all positivc.

(iu) The determinants of叫]lower orders (submatrices) of H(X') that include th巴main diagonal are aIl positive.

Ofthe出ree，only (ii) and (iii) can be practically applied， which is illustrated below.

Example

Minirnize j(XI，x2): (XI -1)2 + (X2 -1)2 -XI X2 (4.38)

O~XI ~3; O S; x2~3 (4.37)

FDC

ヂ=2(xl-2)-x2=O (4.47a)

4.4 ANAL YTICAL CONDITIONS 179

ま=2仇-1)-x1= (4.47b)

EQuations (4.47a) and (4.47b) represent a pair oflinearequations which can be solved as-X1=2，I2=2.The value of the t-unction f is-2.So far only the necessary conditions have been satisfied. The values above can also refer to a point where the

釦nctionis a maximum， or where there is a saddle point.

SDC: For出isproblem [see Equation (4.26)]

日叶什つ

blr

一-H

Is it positive definite?

(i) Not possible 10 test all d.X

(ii) To caJcula~巴 the eigenva¥ues of H，

1=川一λ 一→~I卜=山川一\=山川-3め)仏恥い川-→-¥)り)-1 2-λ川l 、ハ

τ'he eigenvalues ar巴λ=¥，λ= 3 and the matrix is positive definite.

(ui) To calcu¥ate deterrninants of all order富山alincJude the main diagonal and

incJude the element in白efrrst row and first column，

121>0

nu

〉qJ --

2且

d斗

一一'a

司，ゐ・・且ヲu

一The matrix is positive definite.

Sec_ 4_ 4-1.m provides山econfirmation of the numerical values for this example

using Symbolic Toolbox and basic MATLAB commands.

4.4.2 Equality Constrained Problem

Thc problem:

Minirnize f(XI，x2): -XI X2 (4.36)

Su切ectω hl(xl ，x2):xt/4+~-1 (4.39)

O~xl~3; O!:>x2~3 (4.37)


Figure 4.7 illustrated白egraphicaJ solution. It wお noticed山atat the solution， the

gradient of恥 objectivefunction叩 dthc gradient of the constraint we問 paralleland

oppositely directed.Examining other feasible points in FistIm4.7(on the dashed

curve) it can be ascert剖nedthat the special geometrical relationship is only possiblc at the solution. At thc solution a proportional relationship cxists between the gradients

at the solution. Using thc constant of proportionality AI (a positive value) thc relationship between the gradicnts can be expressed as

町=ーλIVhl or Vf+川町 =0 (4.48)

Equation (4.48) is usually obtained in a more formal way using thc method of Lagrange multipliers.

Method of Lagrange: In lhis method， the problem is transformed by introducing an augmented function， called the Lagrangian. as the objective function subject to the same叫ualitycons甘泊nts.The Lagrangian is defined as the sum of the original objectivc function and a linear combination of the constraints.前lecocfficicnts of lhis

lincar combination制官 known創出巴 Lagrangemult伊liers.With rcfcrence 10 thc examplc in山isscction

Minimize F(拘，x2，AI)=f(xl，xu + AI hl(XI，x2) (4.49)

τbe complete problem is developed as

Minimize F(XI，x2，AI) = -XIX2 +λ1(.寸/4+4-1) (4.50)

Subjecl to: hl(XI，x2): 4/4+4-1 (4.39)

O::;;XI ::;;3; 0::;;x2::;;3 (4.37)

Is曲esolution to the回 lsformedproblcm [Equalions (4.50)， (4.39)， (4.37)] the same as the solution to the original problem [Equations (436)，(4.39)，(437)]?Ifthedesign is fi回 sible，也enmost definitely yes. For feasible designs， hl(x" X2) = 0，組d白巴

o均配tivefunctions in Equations (4.36) and (4.50) are the same. If design is nol feasible， thcn by definition there is no solution anyway.

τbe FOC are obtained by considering F(x" X2，入1)as佃 unconstrainedβmctiollin 出evariablcs XI・X2，λl'This provides血reerelations to solve for xj， xi，川

dF i)f • ah，一一=一ι+λ.ー-!.=oax， i)xl

' .'. ax

aF af." ahl 一一=一一+λ，一一=0aA2 ax2" '. aX2

aF 一一=h.=O

(4.51)

4.4 ANAL YTICAL CONOITIONS 181

Equations (4.51) express出eFOCorn国民S釘yconditions for叩巴qualityconstrained

problem in two variables. Thc last equation in thc above set is the COns回 intequation.

百 isensures the solution is feasible. The first two equations can be assembled in

vcctor form to yicld the same information expressed by Equation (4.48)， which was ob凶nedgraphically.百leIeft-hand exp陀 ssionsin山efirst lwo equations abovc are

the gradient of the Lagrangian function:

VF=可+λIVh.=0

Applying Equations (4.51) to thc example of出issection:

aFλIX1 ?一=-x.+一七ム=0oX1 L

aF ?一=-x.+ 2λx.=o ".:¥__ --""2 ' "'""¥olA2 VA2

h144-l=o

(4.52)

Equations (4.52)陀 present伽田equationsin伽田 variableswhich should determine

the values for xj. xi，λj. Note白紙 Equations(4.52) only define the necessary

conditions， which means the solution could be a maximum also. Thcre is one problem with respect to the set of Equations (4.52). Th巴equationsare

a nonlincar set. Most prerequisite courses on numcrical methods do not attempt to

solv巴anonlinear system of equations. Usually. lhcy only handle a single one through

Newton-Raphson or the biscction method. In fact， NLP or design optimization is primarily about tcchniques for solving a system of nonlinear叫uations，albeit of spωific forms. This means Equations (4.52) cannot be solvcd until we advancc釦rtherin由iscourse. Fortunately， having a tool Iike MA札 ABobviates this difficulty， and provides a s位。ngjustification of using a numerical/symbolic tool for supporting白巴

development of the course.2

MA礼ABCode: In MATLAB. there are two ways of solving Equations (4.52)， using symbolic support functions or using thc numerical support functions. Both procedurcs havc limitations when applicd to highly nonlinear functions. The symbolic function is

solveand血巴numericalfunction isfsolve. The numerical tcchnique is an iterative one and requires you to choose an initial gucss to start the procedure. Quite often sevcral

different guesses may bc隠quiredto find出csolution. Solutions of systcms of

equations are fundamental to the rest of出ebook. The following code is included悩

a hands-on exercise. It is avaiIable as Sec4_ 4_2.m. It requires eqns4_ 4_2.m to executeβolve command. Thc co蹴附of血efilcs are listcd bclow

111C問 nrcolher 10018 besldcs MATLAD such田 MUlhcnd副Id臥 ccl.

182 NONllNEAR PROGRAMMING

S町 4_4_2.m

者 Necessary/Sufficientconditions for 者 Equalityconstrained problem 告

者 Optimizationwith MATLAB， Section 4.4.2 % Dr. P.Venkataraman 者

者 Minimizef(X1，x2) -x1 者

者ー・ー----ーーーーーーーーー---・・ーーー・

者 symbolicprocedure もーー・ー--ーーー"ーーーー--ーー'ーーー・・

者 definesymbolic variables format compact . syms x1 x2 lam1 h1 F 者 defineF

F -x1*x2 + lam1*(X1*X1/4 + x2*x2 ・1); h1 x1*x1/4 +x2*x2 -1;

者ヒhegradient of F grad1 ~ diff(F，X1); grad2 = diff(F，X2);

者 optimalvalues

も sa七isfactionof necessary conditions

[lams1 xs1 xs2] sOlve(grad1，grad2，h1，'X1，x2，lam1');

も thesolution is returned as a vector of

者 thethree unknowns in case of multiple solutions も lams1 is the solution vector for lam1 etc.

;mORmT:the results are sorted向 habeti叫 ly

者 fprintfis used to print a string in the 幸 commandwindow

者 dispis used to print values of matrix

f -xs1.*xs2;

fprintf('The solution (x1*，x2合， lam工*， f*)¥n') ， disp(double([xs1 xs2 lams1 f])) ー

曹司・・・ー-------ーーーーー・ーー・ーーー・ーー---

% Numerical procedure 者・ー・・ーーーーーーーーーーーー--ー・・・・ーーー--

も solutionto non-linear system using fsolve も seehelp fsolve 告

者 theunknowns have to be defined as a vector 告 thefunctions have to be set up in an m-file

4.4 ANAL YTICAL CONDITIONS 183

も defineinitial values xinit=[l 1 0.5]'; 者 initialguess for x1， x2， 1am1

者 theequations to be solved are available in も eqns4_4_2.m

xfina1 ~ fso1ve('eqns4_4_2' ，xinit);

fprintf('The numerical solution (x1*，x2*，lam1*):¥n')， disp(xfina1) ;

eqns4_ 4_2.m function ret eqns4_4_2(x) も x is a vector 者 x(l) x1， x(2) x2， x(3) lam1 ret=[(-x(2) + 0.5*x(1)*x(3))，

(-x(l) + 2*x(2)合x(3))， (0.25*x(1)*x(1) + x(2)*x(2) -1)];

Output In MATLAB Command Wlndow

The solution (x1*，x2*，lam1*， f*): 1.4142 0.7071 1.0000 -1.0000

-1.4142 -0.7071 1.0000 -1.0000 -1.4142 0.7071 -1. 0000 1. 0000 1.4142 -0.7071 -1. 0000 1. 0000

Optimization terminated successful1y: Relative function va1ue changing by 1ess than

OPTIONS.TolFun The numerica1 solution (x1*，x2*，lam工*): 1.4141 0.7071 0.9997

The symbolic computation generates four solutions. Only the first one is valid for this problem. This is decided by出eside constraints expressed by Equation (4.37). This is 阻iIIustrationof the a priori manner by which the side constraints affect the problem. On the other hand.出enumerical techniques provide only one solution 10 the problem. 百lIsis a function of the initial guess. Generally. numerical t田 hniqueswill deliver solutions closest to lhe point they started from. The solution is

x;=1.4141; x;=0.7071;吋=1.0

τ'he soh瓜onsfor XI叩 dX2C佃 beverified from the graphical solution in Figure 4.7. The solutions can also be verified through the hand-held calculator. The value and sign for λare usually immaterial for establishing the optimum. For a well-posed problem it should be positive sincc inc陀asingthe constraint valuc is useful only if wc are trying to identify a lower value for曲eminimum. This forces h and f to movc in opposite

184 NONUNEAR PROGRAMMING

directions (gradients). Increasing constraint value can be associated with enlarging the

feasible domain which may yield a better design.

Lagrange Mult，伊，llers:The Lagrange multiplier method is an巴legantformulation to obtain the solution to a constrained problem. In overview， it seems strange血atwe

have to introduce an additional unknown (λ1) to solve the constrained problem. This violates曲econventional rule for NLP白紙 thefewer the variables， the better the chances of obtaining白esolution. As indicated in the discussion earlier，出ELagrangian allows the位ansformationof a constrained problem into an unconstrained problem. The Lagrange multipli巴ralso has a physical significance. At the solution it expresses山eratio of蜘 changein the objective function to the change in臨

constraint value. To illustrate出isconsider:

F=f+λIhl

dF=df+λIdl1

dF. dF. dF dF=一~dx，+ 一一命令+:: aA.， dX

1 _.. . dX2 -'2 • dλ-，..

At the solution， the FOC deems that dF = 0 (which can also be seen in the above detailed expansion). Hence，

λ一一王立 L笠1- dh 611

The above dependence do巴snot affect the establishment of the optimal design. It does hav巴 animportant role in the discussion of design sensitivity in NLP

problems.

General Equallty Constralnecl Problem: Remembering n -1> 0，

Minimize f(X)，匹J'I (4.5)

(4.6) Subjectω[lI(X)]， = 0

X10w SXSXup (4.8)

The augmented problem with the Lagrangian:

Minim取 F(Xム')=f(X)+L入品(X)

k=l

=f(X)+A' h;

=f(め+lTh (4.53)

4.4 ANALYTICAl CONDITIONS 185

Subject to: [h(X)]， = 0 (4.6)

X10w SXSXup (4.8)

10 Equation (4.53)， three equivalent representations for the Lagrangian are shown. The

FOCare

dF νムヨh" (4.54) 一一=ーム+) A." ~'.k =0; i= 1， 2 aJEi axizkh'

Subjectω: 胆(X)]，=o

X10w ~X~Xup

(4后)

(4.8)

Equations (4.54) and (4后)provide the 11 + 1 relations to determine恥 n+ 1 unknowns xヘλ・.Equation(4.8)is used after the solution isobtained-if necessaly.Equation

(4.54) is also expressed as

nu --川川什川山

-hH

V

内，ゐ

ιH V

'n nv r

・・L+

，J V 一一EE

V

(4.55)

Second-Order Conditions: At恥 solutiondetermined by山 FOC，the function should increase for changes in 6X. Changes in企Xare not arbitrary. They have to

satisfy曲elinearized equality cons町aintat the solution. 1t is taken for gr血旬d出atthesufficient conditions Me usually applied in a small neighborhood of the optimum. Also，changes are contemplatedonly with reSPECt to X and not with respect to thE Lagrangemultiplier-TKLagrangemethodis obn calledthe methodqfundetermined coefficient.ーindicatingit is oot a variable. 1n the叩 a叫ly凶“c叫叫d白eri巾va副6ωo叩noft白heFOCfor帥:pp州 emπm凶l恥 Lagrangia釘I刊 wasc∞on帥叫 u聞附n即c∞o∞ns蜘 i削 B勘0叩r町r側 n暗削gf，加m削『町m川nt白he

previouss記巴c叫凶ω0叩n(凹 c∞on附stra凶ine叫dmi加n泊回1由i訟za副t“ion叫)，血巴SOCc叩 beexpec飽dto satisfy the

following relations:

岬 =F(侭X*+必訓)ト川-イF尺'(X.)乃乍)=VF(賦'(X'噂) → 邸d凶T可[内v炉内z守頓怖2F(X'駅限献附(αぽ附削X')ワ羽)]瓜〉刈O 糾

VhTAX=O

In the above [V2F(X勺listhe Hessian of the Lagrangian，with respect to the deSIgn variables only evaluatdat the solution.Also，the F0C mquimthatVF(X・)=0.With reference to two variables and one constraint:


t:..F=核吋+2三走州向)+言叶〉。

M=i際問+2tL位)+詞吋〉。 (4.57)

t:..xl dh/ox2 (4.58)

t:..x2 dh1/dx1

Substitute Equation (4.58) in Equation (4.57) and the SOC requires that the expression in braωmust be positive.百四 derivatives in Equations (4.57)叩 d(4.58)師

evaluated at the m泊imum.

Applying the second-order condition to the example of白issection is 1巴ftお叩

exercise for the陀 ader.Compar芭dto白eSOC for the unconstrained minimization

problem， Equations (4.57) and (4.58)町宮 not側 syωapply，especially the substitution of Equation (4.58). From a practical perspectiv巴批SOCis not imposed for叫ualityconstrained problems. It is left to the dcsigncr to ensure by other means白川出esolution is a minimum solution.

4.4.3 Inequallty Constrained Optlmization

Theproblem

Minimize f(XI・Xz}:ーXIX2 (4.36)

Subject to: gl(XI，x2): 20 XI + 15 X2 -30 S 0 (4.40)

g2(xl ，xz):x114+~-1 SO (4.41)

OSXI S3; OSX2S3 (4.37)

The number of variables (n = 2)剖 dthe number of inequality constrョints(m = 2) do not depend on each other. In fact. an inequality cons釘ainedproblem in a single

variable can be usefully designed and solved. Section 4.4.2 solved the叫ualityconstrained problem. If the above problem can be回 nsformedto an equivalent cquality constrained problem.出enwe have found the solution.百1estandard

回 1sformationreq凶resa slack variable zj for each inequality cons凶 ntgj・UnlikeLPproblems， the slack variable for NLP is not restricted in sign.τberefore， the square of the new variable is added to the left-hand side of the corr四 pondingcons住aint.Thls

adds a positive value to the left-h叩 dside to bring the cons回 intup to zero. Of course a zero value will be added if the constrllint is alrelldy zero.

1週目 ;1

4.4 ANAL YTICAL CONDITIONS 唱87

Transformatlon to sn Equality Constrained Problem

Minimize j(XI・X2):-XI X2

Subjectlo: gl(XI'Xz} + zi: 20xI + 15~ -30 + zi = 0

g2(X.，X2) +這:xi/4+x~-1 +z~=0

OSXI S3; OSX2S3

(4.36)

(4.59)

(4.60)

(4.37)

There are four variables (XI.X2. ZI. Z2) and twoequality cOllstraints. It is a valid equa1ity cons回 inedproblem. The Lagrange multiplier m巴thodcan be applied to this 岡由mau--Tod叫nguish曲emultip目的制ociatedwi出 i叫叫itycons凶脳

血esymbol ~ is used. This is s凶ct1yforcI回ty.

Method of Lsgrsnge: The augmentcd function or the Lagrangian is:

Minimize

F(x 1 ・~， ZI ・ Z2・ß I'ß2) =f(xl.X2) + s1[gI(X1，x2) + zil + s2[C2(XI'ろ)+z~] (4.61)

If the Lagrangian is considered鎚 anunconstrained objective function. the FOC

(necessary conditions) are

dF df. /l ogl . /l dg2 一 =-41一 +s一 =0axl axlaX12axl

dF of. /l ogl . /l ~g2 一=ー+ßI~Ul + s2ー =0dX2

dX2' I"'.dx2 . r~ðx

dF _n

τ一=2s，z，=0aZ1

dF .n

?ー=2s.z.=0aZ2

dF ?

ー =g.+zf=O dPI ul

iJF ? ー=g，.+z~=Ods

2 U "，必

(4.62a)

(4.62b)

(4.62c)

(4.62d)

(4.62e)

(4.62t)


Equations (4.62e) and (4.62t) are珂ualityconstraints. Equation set (4.62) provides six 句uationstosolveforx1，xE，z;，Z5.91.向.By simple recombination， Equ副ons(4.62c)to (4.620 can be collapsed to two equations， while the slack variables ZI and Z2 can be eliminatedfrom tlze problem.

First multiply Equation (4.62c) by ZI・Replacezt by -81 from Equation (4.62e). Drop the negative sign as wellωthe coefficient 2 to obtain

。18.=013282 =0

(4.63a)

(4.63b)

Equation (4.63b) is obtained by carrying out similar manipulations with Equations (4.62d) and (4.620. The FOC can be restated as

dF df. n d8. . n d82 一=;J + ßI~+ 132ー =0dx. dx. r1dx.' r~ðx.

dF dr ~ d8 I ~ d8今

一=二::L+ßI~+ 132ー=0dX2 dX2 r1dx2 r~ðX2

1318. =0

13282 =0

(4.62a)

(4.62b)

(4.63a)

(4.63b)

These four equations have to be solved for xj， xi，日j，s2・Note伽 tzi， Z2 are oot being determincd-which suggests白紙白eycan be discarded from恥 problcma1togc白er.

It would be useful to pretcnd z's ncvcr existcd in the first place. Equations (4.63) lay out a dcfinite feature for a nontrivial solution: cither sj is zero

(and 8/手0)or 8/ is zero (島刊).Since simultaneous cquations蹴 beingsolved，出econditions on曲emultipliers and constraints must be satisfied simultaneously. For Equations (4.63)，出is甘組slaωsinto白efollowing four cases. Thc information on 8 in brackets is to cmphasize an accompanying consequence.

Case a: s. = 0 [8. < 0]; 132 = 0 [82 < OJ

Case b:仇=0[8.<0];仇*0[82 = 0]

Casec: 131 *0 [8. =0]; 132=0 [82<0)

C邸巴 d:日I棋0[8.=0);九*0[82 = 0] (4.64)

In Equation (4.64)， if sj * 0 (or corresponding 8/ = 0)， then the corresponding cons位aIntis an equality. In the prcvious scction a simple reasoning was usedωshow

白鉱山csign of出em叫tipliermust be positive (> 0) for a well-formulated problem. While the sign of the multiplier was igno問dfor白eequality constrained problem， it is included as part of the FOC for出巴inequalityconstrained problem. Before restating

l伺目訂

4.4 ANAL YT'CAL CONOtnONS 189

Equation (4.64)， the Lagrangian is reformulated without the slack variablesぉ (wearegoing to pretend Z never existed)

Minimize F(x.， x2.s ，.132) = .f{x.・x2)+ s.[g.(x..x2)] + s2[8T，.X2)] (4.65)

which is the same formulation in Equation (4.53). The slack variable was introduced to provide出巴 transformationto an equ叫itycons回 int.It is also evident白鉱山e

cons凶 ctionof由巳Lagrangianfunction is insensitive to白巴 typeof constraint Since 白emultipliers tied to the inequality constraint are required to bc positive， while those corresponding to the equality constraints are not， this book will continue to distinguish between出巴 multipliers.This will serve to enforcc clarity of presentation. The FOC for the problem:

e)F dr ~ d8 I ~ d8内

一=二L4144H=odx， dx，' I""'OXt . I""Zdx.

dF df. n dg. . n dg2 一=ー+ß I"\~I + 132一 =0dXz dx2' 1""' d.Xz . I""Z d.Xz

Case a:払=0[gl <0];日2=0 [g2 < 0]

Case b: 13， = 0 [8. < 0];仇>0 [g2 = 0]

Case c: s. > 0 [g. = 0]; s2 = 0 [g2 < 0]

Case d: s. > 0 [8. = 0];島>0 [82 = 0]

(4.62a)

(4.62b)

(4.64)

Equation set (4.62) and any single case in Equation (4.64) provides four equations to

solv巴forthc four unknowns of the problem. All four scts must be solved for the solution. The best design is decidcd by scanning the several solutions.

Thc sign of the multiplier in the solution is not a sujJicient condition for the 泊叫ualityconstrained problem Thc value is un加 portantfor optimization but may be relevant for scnsitivity analysis. Generally a positive value of由巳multiplierindicatcs 白紙 thesolution is not a local maximum. Formally verifying a minimum solution req凶偲sconsideration of the sccond dcrivative of thc Lagrangian. In practical situations. if thc problem is well defined， thc positive value of山cmultiplier usually

suggcsts a minimum solution. This is used cxtensivcly in thc book to identify the optimum solution.

Solutlon of the Example: Thc Lagrangian for the problem is defincd as

F(x.x2.s，.s2) =-x..Xz + s，(2Ck. + 15Xz -30) +九(0.おi~+ .s -l)

TheFOCare

190 NONLINEAR PROGRAMMING

dF Z;=42+2ORI+05h=0

(4.66a)

dF 55=41+15Pl+2h=0

(4.66b)

s1(2OxI + 15Xz -30) =0

s2(0おイ+4-1)=0

(4.66c)

(4.66d)

The side constraints制官

OSX1 S3; OSx.zS3 (4.37)

Case a: sl = 0; s2 = 0: The solution is町ivialand by inspection of Equations (4.66n) and (4.66b).

X1=0; x2=0; 1=0; gJ=ー30; g2 =ー (4.67)The inequa1ity con抑制tsare satisfied. The side con抑 aintsare削 isfied.The values in Equation (4.67) rep回 senta possible solution.

The solutions for the other cases釘'cobtained using MA11.AB・百lecode is available in Sec4_4_3・m.For Cases b and c the appropriate multiplier is set to zero and the resulting thrEe Equations in three unbEowns are solved.For casE d the complete set of four equations in four unknowns is solved.cased is also SOlved numerically.明leMATI.AB code will a1so run code Flg4_8.m which contains th巴commandsto draw山efigure. The output from the Command window is patched be10w.

The solution *** Case a ***(x1*，x2*， f*， 91， 92) o 0 0 ・30 ・1

The solution 企**Case (b)合** (x1*， x2合， b2* f*， 91， 92): 1.4142 0.7071 1.0000 -1.0000 8.8909 0

-1.4142 -0.7071 1.0000 -1.0000 ・6日.8909 0 -1.4142 0.7071 -1.0000 1.0000 -47.6777 0 1.4142 -0.7071 -1.0000 1.0000 -12.3223 0

The solution *合安Case (c)*** (x1*，X2*， b1* f*， 91， 92): o .75001.oo oo o .0500-o .7500o o .14o 6

The solution ***Case(d}***{xl*，x2*，bl*b2*f*，gl，g2): 1.8150 ・0.4200 0.0426 ・1.4007 0.7624 0 0 0.8151 0.9132 0.0439 0.0856 -0.7443 0 -0.0

Maximum nu

曲一offunction evaluations exceeded;

lncrease options.MaxFunEvals Optimizer is stuck at a minimum that is not a root Try a9ain with a new startin9 guess The numerical solution (x1合，X2合，b1*，b2*):

0.8187 0.9085 0.0435 0.0913

4.4 ANALYTICALCONDITlONS 191

The solution for Case a was discussed earlier and is conrtrmed above. Case b hns four solutions. The first one is unacceptable because constrnint gl is not

satisfied. The second solution is feasible ns far as the functiona1 constraints g， and g2 are concemed but they do not satisfy the side constraints. The third and fourth solutions are unacceptable for the same reason. Thus. Case b is not nn optimnl solution.

Cnse c is unacceptable because constraint g2 is in violation. Cnse d hns two solutions the first of which is not acceptable for several rensons. The

second solution sntisfies nll of the req凶rements

• It sntisfies the constraints. 80th constmints釘宮 nctive.

• The multipliersむepositive (maybe a sufficient condition).

・Itsatisfies the side constraints.

百lesolution is

x1=0.8151; X2 = 0.9132; 仇=0.0439; 九=0.0856;

1=ー0.7443; gl = 0; g2 = 0

This is nlmost confirmed by出巴 numericnlsolution which nppears to hnve a problem wi出 convergen田.It is almost at the solution. 1t must be noled here that the MATLAD function used to solve白巴problemis part of the standard package. The functions in the Optimization Toolbox will certain1y do better. Nevertheless. the incomplete solution is reported below but will be ignored due to出esymbolic solution available in Case d.

x，=0.8187; x2=0.9085; s，=O.似 35; 弘=0.0913

???勺Ens憲司ど一

There are now two candidates for白esolution:出etrivinl solution in Case a and the solution in Cnse d. The solution in Cns巴dis favored ns it hns a lower objective function va1ue. All of the cnses above can be explored wi由民spectto Figure 4.8. It is an excellent facilitator for comprehending the cases and血巴solution.

4.4.4 General Optimlzatlon Problem

τ'he genera1 optimization problem is described in the set of Equations (4.1)ー(4.8).百lespecific problem in this chapter d巴finedin Section 4.3 is

Minimize j(Xl拘):ー'X，X2 (4.9)

Subject to: h1(Xl，xV: 20 x， + 15 X2 -30 = 0 (4.10)

gl(X，，xV: <4/4) + (~)ー IS0 (4.11 )

OSXl S3; OSX2S3 (4.12)

叩i111114I11Jjq

193

(4.3)

4.4 ANAL YTICAL CONOITIONS

島;(XIoX2・.. . ，x，J S O. j = 1.2. . . . • m

NONUNEAR PROGRAMMING

議長野忠ZAiil端部出;;355192

(4.4) ~ ~ XI ~.xl' j = 1.2. . . . • n

The Lagrangian: 出;雪Mu/i附附od:The problcm…sfi…y minimizing thc

Minimize

F(x1・・・・，Xn，A.，...，λ/. sI' . . . . srn) = F(XJ，X2.A...sU = -X .X2 +入1(20x.+ 15x2 -30) + s.(0.25x? +xi -1)

Minimize

Subject 10: (4.70)

There are n + 1 + m unk.nowns. The sarne number of cquations are r.叫uiredto solve

出eprob1em.百】ese釘"eprovided by the FOC or出eKuhn-Tucker conditions:

f(x，・・・..Xn)+λ，h， +・・・+入内+s.g，+・・・+srnCm"1(XJ，X2): 20xI + 15x2 -30=0

OSX2S 3 OSXI S3;

(4.71) i=1.2.....n

11 cquations are obtained as

dF df. ~ dh，" . ， dhl . n dg，. . n dg 一=ー+A.，-:;ート・+入ー+PI--+ー・+sm-:m

=0; dx， dx，" -'， dx， ---'dx， -r， ax， -rm ax，

(4.68a)

(4.68b)

aF Z;=-X2+2瓜I+O.5s内 =0

aF à~ =-x.十叫+明x2=o

TheFOC

l叫uationsar芭obtaineddircctly也ruugh山ecqu凶ityconstraints

aF 瓦-=2伽 1+ 15~-30=0 (4.72)

m cquations are app1ied through出巴 2mcases. This implies白紙出ereare 2m possib1e

solutions. Thesc solutions must include Equations (4.71) and (4.72). Each cas巴setsthe multiplier島or由巳 correspondinginequality c叩0叩n附st釘m凶ain蹴t身がtωoze巴瓜 I百ft血he引ml叫t

set to zero.出en出ecorresponding cons位置intmust be feasib1e for an acccptable solution. If出econstraint is set to zero (active constraint). then the corresponding multiplier must be positive for a minimum. With出isin mind出cm equations can b官expressed as

k= 1.2.. ...1 hk(xI'X2• ・.. .Xn) =0; (4.68c)

(4.69a)

(4.69b)

32;:町立は;;zzuyf肌 ωa同 uirωthesoluuo山 system

結滞在i結成23222!な1223;

gl <0

g. =0

s， =0;

s，>O; Case a:

Case b:

(4.73)

If con似 onsin Equations (4.73) are not met.出edωign is not acceptab1e. In implementing Equation (4.73). for each case a simu1taneous to凶 ofm va1ues and cqualities must be assigned. Once these FOC conditions determine a possible solution. thc side constrainlS havc to be checked. As cvidenced in出cexamp1es ear¥ier. this is not built into出巴 FOC.lt is on1y conftrmed after a possib1e solution has been

identified. Equations (4.71)ー(4.73)are referred to as the Kuhn・Tuckerconditions[see Exarnple 4.3 (Section 4.5.2) for additional discussion of the Kuhn-Tucker conditions].

then gj < 0

鳥>0n

pu

出

向

Unu

--=

鳥

め

げ

ば

u

↓

nu --

.，J

eo dd

白U

・

設立制思議部議悲鳴;:(4.1)

(4.2)

f(X.，x2・・...x，J

"k(XI，X2... ..xn)=O. k= 1.2... ..1 Su町民lto:

Minimize

194 NONlINEAR PROGRAMMING

4.5 EXAMPLES

Two exarnples are presented in this section. The first is an unconstrained problem白紙

has significant us巴indata reduclion. Specifically， it illustrates the problcm of curve fitting or regression. The second is thc bearn design problem explo閃 din Chapters 1 and 2.

4.5.1 Example 4.2

Problem: A set of y， Z data is given. It is neccssary to find the best straight line 出roughthe data. This exercise is termed c町 vefitting or data陀ductionor regression. Tok田 Pthis exarnple simple the data points are generated using the parabolic equation Z = O.5y2. The data are generated in arbitrary order and nonun丘ormintervals.

Least Squared Error: The objective function白紙drivesa large number of curve fittinglregression methods is the minimization of the least squared error.百leconstruction of the objective問中irestwo entitiω-the data and山 typeof mathemalical relationship betw回 nthe data. In this巴xarnpleit is a straight line. Generally it can be any polynomial， or any function ror山atmanner. This is usually the way corrclation equations are detcrmined in ex.perimcntal fluid dynamics and heat transfer. For this ex創nple血edata are the collection of points (Y;・ Z/)，i = 1，2， • • • ，n Th批1施ee.郎xp戸ec也凶dstraight Iin巴c叩 b恥巴 c仙h】a紅ra低C~隠.er由i包zedb句y

Z=X1Y+Xz

where xl is the slope and X2 is出eintercept. Using optimizing terminology， there are two design variable渇.For a cubic polynomiaI， there wil¥ be four design variables.百

Zpj is血巴 fittedvalue for也eindependcnt variable Y;，出eobjective function， which is the minimum of thc squ紅 eof the error ovcr aIl of the data points. can bc巴xp閃 ssedas

n 11

Minimize f(x1内)= L [Z/-zpi = L [Z/ー(X尚+X2)]2 (4.74)

;=1 1=>1

It is possible to complete恥 formulationwi出 sidecons凶 ntsaI出ough白紙 isnot necess釘y.明leFOC ar芭

ま=土2[Zj-(x (4.75a)

ま=;2IZi-仰 i叩 (4.75b)

4.5 EXAMPLES 195

Exp叩 dingthe expressions in the brackets and reorganizing as a line釘 ma住ixequ副on

11 " 11

ヱyf2，Y; I~ 司 ILZ;Y;

LY; L11凶 Iiz; /=1 I=J ;=1

Not巴thatthe matrices can be set up easily and solved using MATLAB. The Hessian matrix is the squむematrix on the left The SOC requires出atit be positive definite.

Code: Sec4_5_1.m: Exarnple 4.2 is solved using MATLAB佃 dthe results from出eCommand window are shown below. Compare this code with出atin Chapter 1. The results are displayed on a plot (not included in由巳book).New features in出巴codeare

the use of random numbers， using the sum command， and writing to the Command window. From the plot and山巴tabularvaIues it is clear that a linear fit is not acceptable for由isexample. The quaIity of the fit is not undcr discussion in山is巴xarnpleas it is apparent白紙 anonlinear fit should have bcen chosen. This is lefl as an exercisc. Thc following will appear in山eCommand window whcn出ecode is run.

Results from Linear fit objective function: 16.5827 design variables x1， x2:

2.3458 -1. 5305 yi Zl. Zp diff

4.7427 11. 2465 9.5951 1. 6514

3.7230 6.9304 7.2031 -0.2727

4.3420 9.4266 8.6552 0.7714

0.3766 0.0709 ー0.6470 0.7179

2.4040 2.8895 4.1089 -1.2193

4.1054 8.4272 8.1002 0.3270

4.1329 8.5404 8.1646 0.3757

3.3369 5.5674 6.2973 ー0.7299

2.7835 3.8739 4.9991 -1.1253

4.2032 8.8334 8.3296 0.5039

3.1131 4.8456 5.7723 -0.9267

3.1164 4.8561 5.7802 -0.9241

3.2415 5.2538 6.0737 ー0.8199

0.1954 0.0191 -1.0720 1.0911

0.7284 0.2653 0.1781 0.0871

4.5082 10.1619 9.0450 1.1168

1.9070 1. 8183 2.9430 -1.1247

0.4974 0.1237 -0.3638 0.4874

196 NONlINEAA PAOGAAMMING

1. 9025

0.1749 1.8098 2.9325 -1.1227

0.0153 -1.1203 1.1356

A

X

--r

↑l

・a

M

Fムo

s

e

u

、4FE

q

4

a

7

2

V

0

4

n

4

5

e・

9

5

4

・10

e

2

4.5.2 Example 4.3

This exmnplc is the same as Example 23.the flagpole pmMem.It is quite difficult to

器部諸説3315日程盟百総itzi締結鵠F限切cz

Minimize f(xl占)= 6.0559 E 05(x~ -x~) (4.76)

Subject to:

gl(X内 ):7.4969E05.x7+ 40000x1 -9.7418E06(x1-x~) s 0 (4.77a)

g2(XI'ろ):(5卿 +1.4994E 05xr)(.x7 + X内吋)ー 1・7083E07(Xi-x~) S 0 (4.77b)

g3(Xl'x2):1.909IE-03xl +6.1116E-ω-0.05(x1-xi) S 0

g4(XI'~):X2 -x1 + 0.001 S 0

(4. 77c)

(4.77d)

(4.78) 0.02 S x1 S 1.0; 0.02 S x2 S 1.0

K駒'uh加n-7i町'ucke.町rマω'ondii削胸N陶ons釘: F，而ou町叫rLa砲grang酔cmL机t刷 pli附釘ersa蹴r陀川泡i加n附n則Ir刷r血珂u油tザyc∞o由 t佐reU創nt匂s佃 dt白heLagra加n沼頃gian叩nis

F(X，s) = 6.0559 E 05(xf -~) +

sp .4969E 05xf + 40，∞OXr -9.7418E06(x1-xi)) +

132((50∞+ 1.4994E 05xl)(xf + x内+~)ー 1.7083E07(x1-4))+

臥(1.9091E-03x.+6.1l16E-ω ー0.05(x1-4) +

s4(X2 -x. + 0.001) (4.79)

4.5 EXAMPLES 197

I~I where X = [X¥o X2 ] and s = [131， 132，仇， s4]T.百leKuhn-Tucker conditions require two

叫uationsusing the gradients of Ihe Lagrangian wi出向spectlo白edesign variables 429昨日士山EpdpfJSEJ当

aF _ aF 一一=0:一一=0aXr ・aX2

(4.80)

The acωaI expression for白egradients obtained from Equation (4.80) is left as叩

cxercise for the student.百leKuhn-Tucker conditions are applied by identifying the

various cases based on the activeness of various sets of constraints.

For this problem n = 2 and m = 4. Thcre are 2削=24 = 16 cases that must be

investigatcd悩 partof出巴 Kuhn・Tuckerconditions given by Equation (4.73). While

some of these cases are trivial， nevertheless it is a formidable task. This problem was

solved graphically in Chapter 2 and the same graphical solution can be exploited 10

identify the p紅ticularcase白紙needsto be solved for the solution. Visiting Chapter 2

and sc叩 ningFigure 2.9， it c釦 beidentified白atcons回 ints81 and 83 are active (as

confirmed by出ezoomed solution in Figu問 2.10).The solution is xT = 0.68 m and

xi = 0.65 m. lf 81 and 8J are active cons凶附，白E叩n由恥emL刷 u山l

By thc sa邸mereasoning the mu凶ltipμli旬ersassociated w別i由出巴 inactive constraints 82叩 d

E伽4恥'伽t山i旬S，132 and 134， must bc set to zero. This information on the active constraints can be used to solve for xj， xi鑓白isis a sy山 moftwo叫uationsin two unknowns.

Thisd脚 not， however， complete印刷sfactionof白eKuhn-T山 kerconditions. 131

and sJ must be solved and verified that they are伊副ve.82 and 84 must be evaluated and verified that出eyare less than zero.

Sec4_5_2.m is the code segment血atis used to solve白巴particularcase discusscd

abovc. Thc problem is solvcd symbolically. Two vcrsions of the problem are solvcd:

the original problem and a scaled version.百1巴objectivefunction (4.76) is lJlodified

to have a coefficient of unity. This should not ch回 gethe value of the design variables

(why?). You can experiment with恥 codeand verify if白isis indeed凶 e.Note白cvalues of the Lagrange multipliers during血iscxercise and infer the relation between

the m凶tipliersand the scaling of血巴 objectivefunction. The following discussion

assumes that the code has been run and values available. Referring to山esolution of thc casc above， the optimal va)ues for山巴 design

variables are xj :::: 0.6773 m and xi = 0.6443 m. This is very close to the graphical solution. Before we conclude白紙thesolution is achieved， take note of the multipliers: 向=5.2622e -010叩 d向=-0.1270. If the multipliers are negative，白enit is not a

minimum. The values of由econstraints at the optimal values of the design are 82 =

-4.5553e + 007 and 84 =ー0.0320.Both values suggest that the cons凶 ntsare

inactive. At least出esolution is feasible-all cons回 intsare satisfied. The value of

st is zero and it is a1so an active con

.，

同}


considered active.百1esolution， however， violat岱 theside cons往aints.τ'hegraphical

solution is rather explicit in identifying the minimum at出epoin包 establishedin the

first solution.百1e凶vialcase of the multiplier being zero when the constraint is active

is observcd with respect to the first two constraints白紙 expressstress relations. ln general，出ctcrms in these inequalities have large orders of magnitude， especially if

there are oth巴requations出atexpress constraints on displacements，制芭as，or the like.

Large values exert a large influence on the problem so much as to cause lower value constraints to have little or no signifi四 nccfor曲eproblem. This is usually a severe problem in numerical investigations. The remedy for白isis scaling of the functions.

Scallng: Considcr出eorder of magnitude in Equations (4.77a) and (4.77c). Numerical calculations are合ivenby larger magnitudes. lnequality (4.77c) will be ignored in relation to the othcr functions even though the graphical solution indicates

白紙おisactive.百世sis a frequent侃 curren田 inall kinds of numerical t配 hniquω.Thestandard approach to minimize白eimpact of large variations in magnitudes among

diffe悶 1tequations is to normalize出erelations. In practice this is also extended to出evariables.百 isis referred to as scaling lhe variables and scaling the funclions. Many current software wi11 scale the probl巴mwithout user intervention.

S伺 lingVariables:百1eprcsencc of side constraints in problem formulation al10ws a natura1 dcfinition of scaled variabJes. The user-defined upper and lower bounds are used to scale each varlable between 0四 d1. Therefore，

ふ -X~え=→--'，;x: = scaled ith variable

Xi-Xi

X，=x;(.ヰ-x:)+X:

(4.81a)

(4，8Ib)

In the original problem Equation (4.81 b) is used to substitute for the original variables after which the problem c叩 beexpressed in terms of scaled variable. An altemate formulation is to use on1y the upper value of the side constraint to sca1e thc design

variable:

si=xi/4e

xj=xj寓

(4.82a)

(4.82b)

While曲islimits山巴higherscaled va1ue to 1， it does not set出elower scaled value to zero. For出eexample of山issection， there is no necessity for scaling the design variables since their ordeJ' of magnitudc is one， which is exactly what sca1ing atlempts to achieve.

Scaling the Constraints: Scaling of the functions in出eproblem is usually critical for a successful solution. Numerical techniqucs used in optimization are iterative. In

4.5 EXAMPLES 199

ωch iteration usually白egr叫ientof the functions at thc current value of the design

variables is involved in the calculations. This gradient， exp陀 ssed凶 amatrix， is ca1led 出eJacobi仰 matrixor simply the Jacobiall. Sophisticated scaling techniques [7，8] employ the diagonal en凶esof this matrix as metrics to scale the respective functions.

Theseen肘ωareevaluated at the starting values of the design variables. The function can also be sca1ed in山esame manner as Equations (4.81) and (4.82). For the former an expected lower and upper values of the constraints ar芭necessary.In this exercisc，出econstraints will be scaled using relations similar to the relations expressed by 珂ualities(4.82). The scaling factor for each cons位回ntwill be determined using白E

starting value or the initial guess for the variables. A starting value of 0.6 for both

design variables is selected to compute白evalues n巴cessaryfor scaling the functions. The scaling constants for the eq凶凶onsare calculated凶

g¥O = 293888.4; g20= 102561.12; g30= 1万70E-03; g40= 1

百1efirst three constraints are divided through by their scaling constants. The last 珂uationis unchanged.百1eobj民 tivefunction has a coefficient of one. The scaled

problem is

Minimize 1イ-4 (4.83)

8.: 2.5509~+0.136lx.-33.148(イ -xj) 壬 O (4. 84a)

If2: (0.似 88+ 1.4619x¥)(イ+x内+ヰ)ー 166.5641 (xi -x~) ~ 0 (4. 84b)

あ:1.0868x. + 0.3482 -28.4641 (x1-x~) ~ 0

C4:X2-XI +0.001 ~O

0.02 ~ XI ~ 1.0; 0.025; x2 ~ 1.0

(4.84c)

(4. 84d)

(4.79)

Sec4_5_2_scaled.m: The code (could have becn part of Sec4_5_2.m) investigates the scaled problem in出巴 sameway伽 tthe original problem was

investiga凶.Primarily，出isexploits由巳informationfrom the graphical soI凶 on.1tis

expected出品tthe solution for the design variab1es will be the samc although the mu1tiplier va1ues are expected to be different. From the information in出eWorkspace

window the optimal values for the design variables are xi = 0.6774 and xi = 0.6445. AIlof帥 constraintsare feasible. The optima1 values of白emu1tipliers have changed.

The問 isno looger a zero multipIier-it has tumed positive. There is still one

mu1tiplier with a negative sign implying the point has not met白巴 Kuhn-Tucker conditioDS. Actually， it should have been anticipated since changing oflhe sigll ofthe multiplier is not possible with scaling.


There is stilI the matter of inconsistency betw田 nthe graphical solution and the

dissatisfaction of the Kuhn-Tucker conditions. Exploring this particular example has

been very torturous but has provided白eopportunity to explore important but related quantities Iikc scaling. Another important consideration is出escopc of the

Kuhn-Tucker conditions. These conditions appかonlyat reglllar points.

Regular Polnts: Regular points [8] arise when eq.uality constraints， h(X)， are P陀sent泊白eprob1em.百leyar芭a1soextended to active in叫回Iitycons住必nts(pseudoequa1ity cons位剖nts).Kuhn-Tuckcr conditions arc valid on1y for regu1ar points.百letwo csscntia1 features of a regular point X. are

• ThcpointX・isfeasible (satisfies alI constraints).

. Thc gradients of the叫ua1ity∞nstr温intsぉ wclIas the active in叫回lityconstraints at X. must form a line釘 independentset of vectors.

In the sca1ed problem of Examp1c 4.3 thc constraints g I and 83 are active. The solution xj = 0.6774 and xi = 0.6445 is feasible. The gradient of g I is Vg. =

[-37.631435.454司T.The gradient of g2 is Vg1 = [-34.3119 30.4786]T. lt appears that the two gradicnts arc a1most paral1cl to each oth巴r.This is also cvident in the

graphical solution. The gradients thereforc are not Iinearly indepcndcnt-Kuhn-Tuckcr conditions cannot bc applied at this point. This does not mean the point is not a local minimum. Th巴r巴isgraphical evidence that it is indeed so.

Kuhn・Tuckerare the on1y available forma1 conditions for recognition of the

optimum va1ues. It is univcrsa1ly applied without regard to regulari.か Someadditiona1 considcrations should bc kept in mind [8].

If equality constraints are present and al1 of白einequa1ity cons釘'aintsare inactive，出enthe points satisfying白cKuhn-Tucker conditions may be

minimum， maximum， or a sadd1e point. Highcr-order conditions are neccssary to idcntify thc type of solution.

If the mu1tipliers of the active inequa1ity constr剖ntsare positive， the point cannot be a 10cal maximum. It may not be a local minimum either. A point may bc a maximum if出巴 multiplicrof an activc in叫ualityconstraint is zero.

REFERENCES

1. Stcin， S. K. Calculus and Analytical Geometry. McGmw-HiII， New York. (1987)

2. Hostetler， G. H.， Santina， M. S.， and Montnlvo， P. 0.， Analytical. Nwnerical. and Computational Methods for Science and Engineering. Prentiω-Ha1I. Englewood CIi飽，NJ. (1991)

3. Molcr， C. and Costa. P. J.， Symbolic Math Toolbox-forωe with MATUB-Users Guide， MathWorks Inc.. MA.

PROBLEMS 201

4. MATLABーDemosSymbolic Tool Box-Introduction. online resource in MATLAB.

MathWorks Inc.， MA.

5. Burden， R. L. and Faires， J. 0.， Numerical 刷物'sis.4th ed.. PWS_KENT Publishing

Company， Boston. (1989)

6. Fox， R. L.. Optimiwtion Methodsfor ElIgineering Design. Addison-Wesley， Reading，

MA. (1971)

7. V飢 derplaats，G. N.， N/lmerical Optimiwtion Techlliques for ElIgineering Desigll.

McGmw-Hi11， New York. (1984)

8. Arora， J. S.，lntroduction to OptimaI Design. McGmw-Hill， New York. (1989)

9. Kuhn， H. W. and Tuckcr， A. W.， Nonlulear Programming. Proceedings 01 the Second Berkeley Symposium on Mathematical Statistics alld Probabi/i.η， J. Ncyman (cd.)， Univcrsity of California Prcss， 1951.

PROBLEMS

(In many of the problems be1ow， you are required to obtain the numerical

solution.)

4.1 Definc two nonlinear functions in two variables. Find thcir solution through

contour plots.

4.2 For thc functions in Problem 4.1 obtain the gradients of the function and UI巴

Jacobian. Confirm them using the Symbolic Math Toolbox.

4.3 Define the design space (chooses side constraints) for a two-variable problcm. Dcfine two non1inear functions in two variables that do not have a solution

within this space. Graphica1ly confirm th巴rcsult.

4.4 Define a design space for a two-variab1e problem. Definc two nonlinear functions in two variables白紙haveat least two solutions within白espacc.

4.5 Define a nonlinear function of two variables. Choose a contour value and draw thc contour. Idcntify a point on thc contour. Calculate the va1uc of the gradient at that point. Draw the gr温dientat the point using the computed value・Calculatc

由巳 Hessi回鉱山cabove point.

4.6 Using the relntion in Equation (4.23)， establish伽 tthe gradient is normal to the

tangent [a physical interpretation for Equation (4.23)].

4.7 Define a nonlinear function of伽民 variables.Choose a point in the design space. Find出egradient of出巴 functionat the point. Ca1culate the Hessian

matrix at the samc point.

4.8 Exprcss the Taylor series expansion (quadratic) of the functionftx) = (2 -3x + x2) sin x about由巳 pointx = 0.707. Confirm your results through the Symbolic Math Toolbox. Plot the original function and出eapproximation.

4.9 Expand the functionftx.y) = 10 (1 -x2)2 + (y -2)2 quadratica1ly about the p~int (1，1). How wil1 youdisplay the information? Draw the contours of the origina1 function and the approximation.


4.10 Obtain the solution graphically forthecase when Equation (4.10) is added to肱problem defined by Equations (4.36)， (4.39)， and (4.37). Obtain出esolution using白eSymbolic Math Toolbox.

4.11 Obtain白csolution to thc optimization problem in Scction 4.4.4 using MATLAB.

4.12 1n Section 4.5.1， obtain the coefficients for a qua合凶Cfit to出eproblem.

4.13 Use an example to show白紙 optimumvalues for the design variables do not cb甜 ge江theobjective function is multiplied by a cons凶t.Prove the same if a constant is added to出efunction.

4.14 How does scaIing of the objective function affcct the valuc of山emultipliers? Usc an cxamplc to infer the 陀sult.How does scaling the constraint affect the multipliぽ s? Check with an example.

5

NUMERICAL TECHNIQUE8-THE ONE-DIMENSIONAL

PROBLEM

τ'he one-dimensional unconstraincd optimization problem is often introduced阻 dsolved numerically in most courses山atdeaI with numerical analysisJtechniques. It appears indirectly， notably as finding the root of a nonlinear equation， which is山esame as satis今ing山eFOC for an unconstrained optimization problem. Two of the pop叫artechniques are白eNewton-Raphson and出ebiscction technique.τ'hey釘Eintroduced in Section 5.2 to provide a comparison with the methods used in optimization. In engineering dcsign山esingle-variable optimization problem is prob曲，lya回viality，but one-dimensional optimization is a criticaI component of m凶tivariabledesi伊 andis discussed in S回凶on5.3. Meanwhile the cIassical root finding methods identified above do not play any significant part in numerical optimization techniques as they are considered computationally expensive and not robust enough over a large cIass of problems. An important considcration in nonIinear design optimization is that during the initial iterations， when you are typically far away 合om白esolution， accuracy c皿 bc位置dedfor speed. In these iterations it is also important to move through to山cncxl itcration， instcad of faltering at白ecurrent one. These.criteria have陀qui悶dthat thc one.dimensional techniques be simple in concept as well as easily implemcntable. Two different techniques， the polynomial approx凶lationand曲egolden section， provide popular support for most optimization softw訂e.Very often the two techniques are implemented in combination. They are discussed in Section 5.2.

203

204 NUME円ICALTECHNIOUE5-THE ONE.OIMENSIONAL PROBLEM

5.1 PROBLEM DEFlNITION

In order to connect with the discussions later， the one-dimensional variable will be identified nsαinstead of X or XI・Inthe fol1owing chapters血isis a1so referred to ns the onc-dimensional stepsize computation.

Example 5.1 This example is similar to one in Chapter 4:

Minimize f(α)=(α_ 1)2(α-2)(α-3)

Subjectto: 0 SαS4

Theproblemdoesnotrepresentanyrealdesign.Itwasconstructedtohavemultiple local minimums within the area of interest. Expression (5.1) reprcsents an

unconstrninedproblem.Thesideconstraintsaccompanyal1problcmsinthisbookto con vey theidea that therearenotru Iy u nconstrained prob lems. Sideconstr ain tsalso define an acceptable design region for all problems. Section 5.2 will explore this prob lemthroughthcvar iou smethodsmen tionedcar Iier .

5.1.1 Constralned One-Dlmenslonal Problem

The only cons~raint that Example 5.1， a single-variable problem， can a，∞ommodate is inequaliηconstraint. From thc previous chaptcr there can be mo隠白血 oncsuch

cons回 int.A single-variable problem cnnnot have如何ualityconstraint (Chapter 4). In a constrnin巴dmultivariable optimization problem it will often be necessary lo.find 白estepsizeαsuch白紙山econs位置intjust becomes active. It is possible to defme出巴

problem.

Example 5.1a

Minimize f(α.)=(αー1)2(α-2)(α-3)

Subject to: g(α): 0.75α2-1.5αーIS0

(5.1)

(5.3)

OSαS4 (5.2)

In actual applications it is more Iikely出atcxpressions (5.1)ー(5.3)establish two

problems that provide different solutions to出cstepsizcα. The first is白esolution to expressions (5.1) and (5.2)， which is the samc as Exnmplc 5.1. The second is the solution to

g(α.):0.75α2-1.5αー1=0

OS αS4

(5.3)

(5.2)

(5.1)

(5.2)

5.2 SOLUnON TO THE PROBLEM 205

The second solution is useful only if expression (5.3) is an active constraint Otherwise

the solution to Example 5.1 wil1 prevai1.百lefinal solution chosen ensu陀 sexpressions

(5.1)ー(5.3)are satisfied. A graphical investigation in山enext section should illustrate

this feature c1early.

5.2 SOLUTlON TO THE PROBLEM

The solution to Examples 5.1佃 d5.1ais examinedgr司phicaIIy.Next the c1nssical root finding tec加 iqucsofNewton-Raphson and山ebisection method are presented. These M芭followedby the polynomial approximation and山egolden section mcthod.

5.2.1 Graphlcal Solutlon

The example has already made its appearance in Chapter 4. Figu閃 5.1is a plot of the

objective function in Example 5.1. It also includes出csolution ωExample 5.1a. If expression (5.3) represents an incquality constraint， the feasible region for one-

variable functions is quite different from the two-vruiable problems encountered in

， J "， '

/

3 g(α)

内'h

{国}回，(司)-

constralned minimum

。

ー1属、.fe絡iþI~ .r~砂舟"

HJ"'lιglobal ~Inlmum unconstrained minimum

、.

;.. --『ーーーーー-2

4 。 2 a

3.5 0.5 1.6

円gure5.1 Graphical solulion: Example 5.1.

206 NUMEAICAL TECHNIQUES ーTHEONE.OIMENSIONAL PAOBLEM

::tz;;ロ:;立とむねなお品321i222211出;宮探知1 FZZ;itmt品川双山口

Example5.1

α事=2.6484; f* =ー0.6195 (5.4a)

Example 5.1a

α* = 2.5275; f. = -0.5815 (5.4b)

5.2.2 Newton-Raphson Technique

The Ncwton-Raphson technique， also rcferrなdto as Ncwton's technique， is a gradient-bascd剣山nto finding山croot of a singlc nonlinear eq剛 on.Root finding

lおおむlニitU122ZZ;::rh刷出eequati川おaぷ。;:;L

中(α)=0

wher官ゆ(α)is a nonlinear equation in the variable α. 百letechnique has additional featurcs:

・ Ithasag回 metricunde甲inning.

• It usωthe Taylor series expanded IinearIy. ・Itis iterative.

・ It has the property of quadratic convergence.

Any iterativc tcchnique addrcssing Equation (5.5) can be summarized as

Generic Algoritlrm (A5.1)

Step 1. Assum巴α

Step 2. Calculate.1α

Step 3. Update a =α+.1α

If converged (中(δ)=:0)出enexit

If not converged (ゆ(匂:;f:0)白enα←ago to Step 2

IpiIω10 be downloaded fwm thc web sile町宮 indicaledby boldfncc snns serif Iype.

(5.5)

5.2 SOLUT10N TO THE PAOBLEM 207

The above sequen田 isfairly standard in iterative techniques for many classes of problems. It also caplures the essence for most of the techniques to follow in this book. Slep 1 indicates the primary fi伺 ωrcof iterative me白ods-thestarting solution. This is an initial guess provided by the user 10 slart the iterative process. The iterative process is continued 10 the solution by calculating changes in the variable (.1α.). The value of the variable is updated. Convergence is checked. If Equation (5.5) is satisfied， then the solution has been found. If convergence is not achieved，出叩 theupdated value provides the value to proceed with出巴 nextiteration. Of course， .1αcan be calculated in many ways and Newton-Raphson is just one of the ways ωestablish the

I change in the variable. 費lj

Calculatlon 01 Aα (Newton-Raphson): Let αbe the current value of the variable. It is assumed that Equation (5.5) is not satisfied at出isvalue of the variable (出e陀asonfor us to iterate). Let a be a neighboring value. Ideally we would like 10 achieve convergence at this valu巴(even邸 weare aware it rnight take us several itcrations 10 achievc converg巴nce).Using Taylor's theorem expanded to the linear terrn only，

dゆゆ(α)= ct(α+似)=中(α)+25Aα=0

Aα=一魁旦1=-'坐Icb(o.) 坐 Idlα|τ

、，

(5.6)

(5.7)

For Equation (5.7) to bc effcctive，山egradient of the function should not be zcro. lt is a1so inevitable出atchanges inαwill be large where ct is flat， and smal1 where the slope is large. For the Newton-Raphson method to be effcctive the iterations should avoid問gionswhere the slope is small. This is a scrious disadvantage of出ismcthod. Figure 5.2 ilIustrates the geometrical construction of tbe techniquc at the point α= 3.

Example 5.1 The Newton-Raphson tcchnique is applied to the first-order conditions (FOC) of Example 5.1.

Minimizc f(α)=(αー1)2(αー2)(α-3) (5.1)

Subject to: 0 SαS4 (5.2)

The FOCis

中刷{ω叫=去=判均αト一 l州 -2引吻州2均伽釧)い州彬ベ(似ωα-3)司)+(似αトい川一→1り)向州勺恥3勾)+(似α十日一→lり)州αrμ-2十2(5.8)

208 NUMERICAL TECHNIQUES-THE ONE.OIMENSIONAL PROBLEM

10

8

6

2

。

-2 0 0.5 1.5 2 2.5 3

α

Flgure 5.2 Development 01 Newton.Raphson technique.

3.5 4

ηJe Technlque: Two iterations are shown here. The complete application is done through MA11AB codesecs-2-2.m.The code uses symbolic calculation.In rumino 悦 codethe figure window displays印刷rnationof the iterati削 ravelω曲目仰60;

Also shown is the change in 6αwith each iteration. The iterative results are summarizcd in a table 8t山ecnd. The ca1culations requirc the gradient of中(α):

ゆ'(α)=2市(α-2)ポ(α-3) +4*(αー1)*(αー3)+ 4*(αー1)*(α-2) +2*(αー1)2

Iteration 1:

α=3; f(α)=0.0; ゆ(α)= 4 It'(α)=16 6α=-0.25

It(α+L1α.) = 0.875-not converged， at least another iteration neccssary

Iteration 2:

α=2.75; f(1α.)=-0.5742; 中(α.)=0.0875; 帆α.)=9.25; 却=ーO.ω46

。(α+6α)=0.1040ー notconverged， at least anothcr iteration nccessary

5.2 SOLUTlON TO THE PROBLEM 209

τ'he code Sec5_2_2.m r珂uiresfive iterations to converge the va1ue of It to

1.0E心8(considered zero here). T悩svaIue of 1.0E・08is termed the convergence (or

stopping) criterion. Thc acccleration of convergence or rate of convergence to thc

solution is formally defincd in many different ways. One of thc ways is to monitor the

ratio ofゆ(α+6α)1中(α).Anothcr isωtrack 6α. ln applying and understanding

numericaI techniques it is important to develop an instinct and feel for the numbers corresponding to various functions 脳血eiterations progress. The speed or白equality

of convergence can be gauged by出eshift in the dccimaI point in the quantity of interest through the iterations. Consider the vaIue of It through the five iterations

[4.0000，0.8750，0.1040，0.0023，0.0000]. Definitely白echangcs in中arenonlinear. The Newton-Raphson technique can be direcUy applied to Equation (5.3)-after

aIl the meUlod is designed to operate on these kinds of problems. It is left as an exercise

for山巴 r回 der.百lereis a very important feature of most itcrative techniques that

exp!ore m叫tiplesolutions-as is白ecase of Example 5.1. Rerun the code

Sec5_Zι2.m with a starting vaIue of 0.5 for a.. After six iterations，αconverges to 出evaIue of 1.0， which is a locaI minimum. Where would iterations lead if the start vaIue is 1.5 ? These answers are回 syto s田 graphicaIlyin a one-or maybe a

two-variable problem. In a typicaI engineering design problem， significant experience and intuition are required to recognize Ulesc situations when出cyoccur. A healthy

S出 picionof the solution must aIways be in the back of one's mind when exploring multivariable problems. In summary， most iterative numerical techniques are designed to converge to solutio，1S that are c/ose 10 where they startfrom.

5.2.3 Bisectlon Technique

This is another popular tcchnique to find曲目∞tof a function. It is also caIled a binary

search or intervaI haIving procedure. Unlike the Newton-Raphson method this procedure does not民quirethe evaIuation of白egradient of曲efunction whose root

is being sought. One of the reasons the method is being included is to compare it with the golden scction method discussed latcr. The numericaltechnique in山ismethod is

basedonthe問おoning白紙therl∞t(or zero) of a function is trapped or bound between

a positive and a negative vaIue of the function. The actuaI solution achieved by血E

method is an intcrval in which出ezero or the solution can be located. Thc finaI interval

is typi回lIydctcrmined to be a vcry smalI value， callcd thc tolerance， of the ordcr of lOE-08. S泊ce出isis a root finding procedure， when applied to optimization it is

applicd to恥 gradientof批 ot訪問tivewhich we identificd asゆ(α.)Equ副on(5.5).

Establishing the minimum of the objective is thereby translatcd to locating曲erootof

出egradient function. It is possiblc to devclop a bisection tcchnique for directly

handling the minimization problem.

The method r珂uirestwo points to start， sayαn and αb・ThevaIue of It at由民E

points must be opposite in sign.τ'hesc can be the sidc constraints of the

one-dimensionaI optimization problem. lt is assumed白紙 atIcast one solution exists

in白isiniti凶interval.During each iteration，出isinterva1 is haIved with出emidpoint

of the interval replacing cither 叫 orαb，while kccping thc root still trappcd bctween

210 NUMERICAL TECHNIQUES-THE ONE-DIMENSIONAL PROBLEM

amction values ofopposite sign.The itemtiveMmique is expressed as the following aIgorithm (a set of procedures白紙紅erepeated).

AlgoritIJm!or Bisection Metllod (AS.2)

Step 1: Choose 叫 and叫 tostart. Let叫〈叫

Step 2: Set α=叫+(叫ー叫)12

Step 3: If中(α.)= 0.0 -Converged Solution -exit

日se If(αb-αn)<lOE・04-tolerance met -exit

Else Ifゆ(α.)*中(叫)>0.0;曲en叫←α

Elseαb←α

go to Step 2

We consider Example 5.1 which is reproduced here

Minimize f{α)=(α_ 1)2(α-2)(α-3)

Subject to: 0 Sα云4

ThcFOCis

中仰(何仲α

(5.1)

(5.2)

(5.8)

S関 5_え.3.m:This地孔ABcode runs the bisection method on臥制ple5.1. It

iIlustrates the trapping of山emullrnum by the two values on either side Of the mot. 百le陀町宮 17iterations for a tolerance of 1.0 E・04which terminated thc program.

Compa削 hiswith 5 itcrati叩sfor the Newton-Raphson 1削 hod.for a bettcr solution. U巴lowernumber of i飽ratJonsc佃 be油凶to恥 quaJityof ir由'nnation山 edto upd脱出ed悶伊 variablechangcτnc gradient-bascd information uscd in thc Newton-RaPIEBon tEChmque ls more sophisticated.though出cgradient computation is an additionaJ work load.

TK problem described by Equation (53)cmbe handled directly by thEbisection 脱出od山町(α.)is恥 same箇中(α).In a wa町Y.t出h附e

o叩pp卯o削 utザYtωo 陀附rev1判1路凶附s討i比tprior knowlcdge. parti 叫 arly with resp 附 ω defining and

剛 slatingan algori伽 intoIUD出ngcode. There is u刷su拘 ast凶ron昭gc∞orre附 s叩po叩nd伽lbetwecn 自estcp-by-stcp d ω cription of 出e numericaJ technique (t山healgorit白hr町同E白山ec∞od也e白伽aωtwill implement 出e 蜘 ps ω effect a solution (for example.

:22tit-B;日EP;521111:ZZ220:よ22LtJbreばfollowlng two methods are used extensively foronedimensional mi凶mization.They handlc the minimization problem directly.

5.2 SOLUTION TO THE PROBLEM 211

5.2.4 Polynomlal Approxlmatlon

The me白odis simplc in concept. Instead of minimizing a difficult function of one

variable， minimize a polynomial出atapproximates the function. Thc optimal vaIue of 曲evariabl巴白紙 minimizesthe polynomial is thcn considcred to approximatc the optimal vaJuc of白evariable for t1Je originaI function. It is rare for t1Je dcgree of t1Je

approximating polynomiaJ to excecd threc. A quadratic approximation is standard 叩 less曲e曲irddegree is w釘ranted.It is c1ear that scrious errors in approximation are expected if出epolynomial is to simulate the behavior of出eoriginal function ovcr a l釘.gerange of vaIues of山 variable.Mathematical t1Jeorems exist白紙 justifya quadratic representation of t1Je function， with a prescribed degree of error， witIJin a smaJl neighborhood of t1Je minimum. What 山isensures is出att1Je polynomial

approximation gets better as白eminimum is being approach叫.Example 5.1.

repf吋 ucedfor conveniepce， is used for iI1ustration of the procedure.

Example5.1

Minimize f(α.)=(α_ 1)2(α-2)(αー3) (5.1)

Subjcct to: 0 S αS4 (5.2)

A quadratic polynomiaJ P(α) is used for the approximation. This polynomial is expressed as

P(α)=bo+bρ+b2α2 (5.9)

Two elcments n田 dto恥 understoodprior to t1Je fol1owing discussion.百lefirst conccms白eevaluation of t1Je polynomi叫.and t1Je second concerns出einclusion of

exp陀ssion(5.2) in出ediscussion. Thc polynomiaJ is completely defined if the coefficients bo• b.， and b2 arc known. To determine恥 m.t1Jree data points [(α1'/.)， (α.2'!z). (α.3，13)] arc generated from Equation (5.1). This sets up a Iincar system ofthree equations in尚昆eunknowns by問 quiring白紙t1JevaJucs of t1Je function and the polynomiaJ must be the same at t1Je t1Jrce points. The solution to t1Jis system of equations is山 valuesof t1Jc coefficients.百1巴 considerationof expression (5.2) depends on t1Je type of one-dimensionaJ problcm being solved. If the one-dimcnsionaI problem is a genuine single-variable design problem，出enexpression (5.2) needs to be present. If 出e one-dimensionaI problem is a subproblem 台。m 山e

muJtidimensional optimization problem， t1Jen expression (5.2) is not available. In that cωe a scanning proccdure is used to defin巴α1，的， andα3・

Scannlng Procedure:百lepr民間 isstarted from恥 lowerIimit forα. A vaJue

ofzero for出isvaJue叩 bejustified since it refers to vaJues 鉱山currentiteration. A constant intcrval for α，6.αis also identified. For a wel1-scaled problem t1Jis vaIue is usual1y 1. Starting at t1Je lower Iimit，曲。 intcrvaIis doubled until tlJree points are

.

213 5.2 SOLUT¥ON TO THE PROBLEM

，，，，，，，，. ，，，，

t ，，，，，，，，，，，，，， r ，

NUMER¥CAL TECHN¥QUES-THE ONE.O¥MENS¥ONAL PROBLEM

detennined such出atthe minimum is bracketed between them. Wi白 respectto

Example 5.1， the scanning procedure generat回 thefollowing values:

212

10

α¥ =0; /¥ =6 α= 0; j{O) = 6;

B 同=1;ん=0α= 1; j{l) = 0;

this cannot be円 ωtheminimum is n01 yet回ppedα= 2; j(2) = 0;

a

2

。

/2 = 18

A process such as that illustrat巴dis essential sS it is both indifferent to the problem

being solved and can be programmed伺 sily.The important requirement of any such proc凶 sis to ensu白山atthe minimum lies between出eIimits established by the procedure.百lisprocedure is developed as MATI.AB m-fi)e UpperBound_1Var.m.

The set of Iinear equations to estabIish the coefficients of the polynomial in Equation (5.9) is

6 = bo + b¥(O) + b2(0) (5.108)

円=4;α= 4; j(4) = 18;

(5.lOb) 0::: bo + b¥(l) + b2(l) 4 3.5 3 2

α 1.5 0.5 。(5.lOc) 18 = bo + b¥(4) + b2(16)

polynomial appro必ma1ion.

∞de development is termed a block structured approach-In this way-larg巴code

development is possible usmgsmallerpieces ofexisting code.

Rgure5.3

(5.11)

Using MATI.AB出esoh瓜onto白eequations is

ho=6.0; b¥ =-9; b2=3

百ICapproximate solution to Equation (5.1) is 1.5. Figu陀 5.3dωcribesthe polynomial approximation procedure. lt is clear出atthe approximation leaves much to be desired. On the other hand， if the datn for the polynomial coefficients were around a smaller

region nearα= 2.6， then出巴resultswould be more impressive. Note出atra山町出m

usc a higher-order polynomial， a better set of data is p児島町吋.

The scanning procedure and出epolynomial approximation techniques will find

significant use exploring multidimensional optimization problems. The s位置tegyfor devcloping MATLAB codc from now on must include出epossibility of code reuse.百lesame code segments can be used in many di仔erentnumerical techniques. Such generic

upperBound1Var.m:This codcsegment implements山edetcnnination of山e

upperboundoilhE凶 onof one variable. The input to the function is the附 ncofthc

器問:思議議部先制抗告。部cotZ詑Z3;42;;;22dEmusthaM山 Bm-fi)e，in白esame directory

tZTftU3231:;:;Z?5512-おおむtwhich thc function

P(α.}=6-9α+3α2

dP 一一=-9+6α=0;α:=α・=1.5 0αr

and the polynomial is

The minimum for the polynomial is the solution to

(5.12)

cod巴 segment implements 白e polynomial

function of one variabl巴・ This function uscs

upperBound_1Var('EXample5_1' ，0，1，10)

PolyApprox_1 Var.m:羽lis

approximation method for a

Usage:

214 NUMEAICAL TECHNIQUE5-THE ONE.OIMENSIONAL PAOBLEM

UpperBound_1 Var.m to detennine the range of出evariable. The input to白efunction is the name of the function; the order (2 or 3) of the approximation; lowbound-the start valuc of the scan passed to UpperBound_1Var.m; intvlstepー恥 scanninginterval passed to UpperBound_1 Var.m; in町ials一郎

number of scanning steps passed to UpperBound_1Var.m. The output of出eprogram is a vector of two values. The first element of the vector is the location of the minimum of the approximating polynomial， and the second is the function value at this location.

The function 陀 ferencedby出ecode must be a MA TI.AB m・file，in the same directory (Examp抱5_1.m).百1einput for Example5_1 is the value at which the function needs to be computed. and its output is the value of the function.

Usagc: Value PolyApprox_1Var(' Example5_1' ，2，0，1，10)(5.13)

The two programs us巴出eswitch/case and the /eval statements from MATLAB. The code also ilIus町atescalling and retuming from other釦nctions.

5.2.5 Golden Section Method

Th巴methodof golden section is thc crcam of the family of interval 問ducingmethods (for example.出ebisection method). It reduces出einterval by thc same fraction with each iteration. The intervals釘ederived from the golden seclion ratio. 1.61803. This ratio has significance泊a巴stheticsas well as mathematics [3，4]. The method is simple to implement. It is indifferent to出eshape and continuity properties of血efunction being minimized. Most important， Lhe number of iterations to achieve a presαibed tolerance can bc established before Lhe iterations start.

Algorithmfor Golden Sectioll Method (AS.3)

Step 1. Choose αlowα叩，

τ= 0.38197 (from Golden Ratio)，

e = tolerance = (L¥α)Onal/(αup_αlow)

N = number of iterations = -2.078 ln e

Sωp2.α1 =(1-τ)αlow+τα叩;fl=/(α1)

向 =τriOW+ (lーτ)α，UP;f2=爪句)

τnc points are eq凶distantfrom bounds

Step 3. If ( i < N)

If/I >/2

α10"←α.;α1←α2;fi←12

~=τα，Iow + (1-τ)α叫ん=/(向)

i←i + 1

GoTo Step 3

Ifh>/.

Il'J

5.2 SOLUTION TO THE PROBLEM 215

αup←α/:2;向←α1;12←Aα1 = (1 -'t)a!ow +ταゆ;/1=/(α.)

; r i + 1

GoTo Step 3

si出iiZ昔話器鴻!日only calふlateふevalues. The algo仙 mis implemented in a different file.

ExampleS.l (5.1) Minimize /(α)=(α_ 1)2(α-2)(α-3)

Golden Section Method

St叩 1.(1.10" = o.o;low = 6;αuP=4;!"'= 18

L¥<Xj!nol = 0.001; E = (0.00114); N = 17

Step 2.α. = 1.5279;/. = 0.1937;向 =2.4721;12=ー0.5401

F1gure5・4i1lustrates th巴layoutof information at this stage.It can bCVErind山atα..α2 are located symm巴町icallywithmspect to outer bounds dO嶋田，"p.

Step 3. i = 1

/1>12 (1.lolV = 1.5279;low = 0.1937;叫=2.4721;/. = -0.5401

α2 = 3.0557;12 = 0.2486

dijjY35233器:

掴

調

i=2

Step 3. continued...

12>13 a.up = 3.0557;戸 =0.2486;α2= 2.4721;五=-0.5401

α1 = 2.1115;12 =ー0.1224

i=3

Go To Step 3 etc

In each itcration Lhere is only one制 ofcal叩lationto eva山a胸中evariable and伽

value ofthe function.The rest ofthEComputauonts masSIgnsng extsting informatIon. The differcn回凶臨ou町 bomdsisalsochangingwitheach kmuon-hExmpleal

Start: L¥α=4.0

Iteration 1: L¥α= 4.0 -1.5279 = 2.4721

217

vector of four pairs of variable and function values after the final iteration. With a small tolerance剖 yone of lhese pairs provides出巴minimumlocation of the function. It is recommended山atthe second pair be accessed for this pu中ose.百1efunction referenced by the code must be a MATLAB m-file， in the same di悶ctory(Example5_1.m). The inpul for Example5_1.m is the value al which the function needs to be computed， and its output is the value of the function.

U阻 .ge:

Value GoldSection_1Var (' ExampleS_1' ，0.001，0，1，10) (5.14)

There are no new commands. Thc organizing and displaying of information in the Command window is worth noting. If this code is part of a larger program，出叩 theprinting of information in the command window needs to be switched off. It c叩 be加medon for debugging.

Compar/son wlth Polynom/al Approxlmatlon: The two popular methods for miniouzing a function of one variable are polynomial approximation or golden section. The algorilhms for polynooual approximation and由巳 goldensection have significant differences. The former is a one-shot approach and is accompanied by significant error in the estimation of the minimum， which will improve as the data for approximation get better. Implied in the implementation of the polynomial approach is the continuity of白efunction. The golden section method， on the other hand， is佃

iterative technique. The number of iterations depends on出etoleranc巴expectedin血efinal隠sultand is known prior to出巴startof the iterations-a signifi伺 ntimprovement relative to the Newlon-Raphson method where山巴 numberof iterations cannot be predicted a priori. The implementation is simple and the results impressive as it is able tohomeinon曲巴oun回1um.It is indifferent to the nature of the function.

There is no reason why the two cannot be combined.τbe golden section can be used to establish four data points with a reasonable tolerance (instead of a low value) and a cubic polynooual can be fit 10 identify the approximate location of minimum.

百1巴 one-dimensionalsubproblem in a multivariable optimization problem is employed for白edetermination of the stepsize after thc search direction has been identified. It is easier 10 undcrstand these lerms by recognizing the generic algo仙 mfor unconstrain吋 optiouzation.In order to focus the discussion， rather than白egeneral objcctive function representation， a specific one， Example 5.2， is introduced.

IMPORTANCE OF THE ONE-DIMENSIONAL PROBLEM 5.3

(5.15) f(xl・X2'X3)= (xlーら)2+ 2{~ -x3)2 + 3(x3 -1)2

Example5.2

Minimize

5.3 IMPOATANCE OF THE ONE.DIMENSIONAL PAOBLEM

5

αup

' t

' E

a

'J

s

NUMEAICAL TECHNIQUES-THE ONE.DIMENSIONAL PAOBLEM

20

18

18

216

14

12

10

8

6

{6}h

4 3

Golden 5ectlon-starting valuas.

dα= 3.0557 -15279 = 1.5278

2

α

Flgur由5.4

Iteration 2:

。

4

-2 -1

2

。

The final iteration is reproduced by running the following code:

iteration 17 2.6395 -0.6197

2.6406 -0.6197

2.6402 -0.6197

2.6399 -0.6197

百leloleranceα2一αIin the final iteration is about 0.001 as expecled.

GoldSection_1Var.m: The codc lr叩 slates白ealgorithm for the golden s田 tionm白od(A5.3) in川弘TLABcode. The input 10 the function is出ename of thc function 俳tnCtname)whose minimum is being sought;曲etolcrance (tol) of the approximation; the 平蹴 value (仰low叫bo側und)of 白恥es釘c叩 p仰a鎚蹴e叫dtωo Up仰p仰er市Bo叫und_1Va町r口r.m宵m舵官n仕1;出肱

巴s鈴ca朗nmn略gi加nt飽erv叫(uωt仰 1)paお55鉛e伐吋dtωoUp仰perBou叩md_1Va町r口'.m;t白hem聞u叩m帥E蹴 p戸sρ仰仰11巾旬 p蹴叫 toUpperBound_1 V町 .m.The output of恥 programis ;

218 NUMERICAL TECHNIQUE5-THE ONE-DIMENSIONAL PROBLEM

Ac山富oryglance at Example 5.2 indicates曲atthesolutionisatxl= l;x2= I;X3= 1; and the minimum value offis O. Three variables are chosen because we ar芭notgoingto use any graphics for illustration (it is essential to follow the vector algebra).

Generlc Algorlthm (A5.4): The generic algorithm is an iterative one and is also refer聡 dtoωasearch algorithm. as the iterations tak巴placeby moving along a search direction. These search directions can be determined in several ways (Chapter 6). The algorithm without any convergencelstopping criteria can be expressed as

Step 1. Choose Xo

Step 2. For each iteration i

Determine search direction SJ vector Step 3. Calculate sX; =αjSj

Note: sX; is now a function of the scalar clj as Sj is known from Step 2

clj is called the step size as it establishes the length of sXj

Xjφ1 =X/+sXi

的 isdetermined by Minimizing F(Xi+l)

百Iisis referred to as one-dim巴nsionalstep size computation

i←i + 1; 00 To Step 2

Applicalion 0/ Generic A駒市hm(A5.4)ωEXIlITlple5.2

…~[g}印刷

、.. ，

nu x

---‘

au rJ

et o

t

n

9uw M

V且e

b

陪

1Ill1

t

1

1

0

0

α

drllML

四

=

ι』苛

EBEBEE--EEEJ

叫副

.006.

grill-L

内u

，.

n

α

・

凶

+

山、引川円引J

L

z

r

E

・E

・-'EEEEEL

-

一

「

間

x

U11111111

蹴

ト

MMMし

，，‘、・・・

d川引J引

Jα

F1111111L=

=

'

J

・

g

u

A

U

ヲ

-

司

3

nrnr

e

e

-

-

p

a

n

a

f(X1) = f(α1) = (0 -0)2 + 2(0 -6α1)2 + 3(6α1一1)2= 72αl+3(6αJ -1)2

Minimizing j{α1)，出cvalue of α1 = 0.1. This is a one-dimensional optimization problem.

While the solution above was obtained analytically because it was a simple linear exp陀 ssion，出epolynomial approximation or the golden section method could have been used inst巴ad.


Xl=lL|… (5.16)

and so on.


h 山issection additional examples of single-variable optimization problems arc explored. The available code for the polynomial approximation and golden section should help solve any and all single-variable optimization problems. This is largely left to the reader to exploit. In this section only extensions， modifications， or creative appIications of the one-dimensional optimization problem ar芭 considered.First， Example 5.2 is revisited to modify the golden section method so出atit can be used to 四 lculate由巳 S旬psizein multidimensional problems. Example 5.3 is a solution to the

Blassius problem.百lisis an example of the exact solution to白eNavier・Stokes珂uationfor flow over a flat platc. Example 5.4 is an examination of the inequality constraint in Examplc 5.la by the golden section method so that equality and inequality constraints can be handled.

5.4.1 Example 5.2-lIlustratlon of General Golden Sectlon Method

τ'he example from血eprevious section is visited again to modify出egolden section method to d巴tcrmine白estepsize for multidimensional problem.百 isbook is also about getting MATLAB to work for you. In a scnse出 ssubsection is largely an exercise in developing the codc白紙willsee considerable use in the next chapter. An明uivalentdevelopment with respect to出epolynomial approximation method is left as an exercise for the student. τ'he goldcn section method requires the establishment of the upper bound.

τ'h巴refore，two functions wil1 need to be changed to handle dcsign vectors and the search direction vectors. The code is availableぉ UpperBound_nVar.m加 dGoldSectlon_nVar.m. The modifications are nOl very challenging and can be inferred by comparing the usage of山etwo functions.

For the single variable:

Usage: UpperBound_1Var (' Example5_1' ，0，1，10) (5.12)

For many variables:

U腿 ge:UpperBound_nVar('Example5_2'，x，s，O，l，工0) (5.17)

x: current position vector or design vector

s: prescribed search direction

220 NUMERICAL TECHNIOUE5--THE ONE-DIMENSIONAL PROBLEM

Of course， Example5_2.m retums the valu巴of出efunction corresponding to a

vector of design variables.百lisimplies白紙thefunction call to Example5_2.m can

only take plac巴afterthe design variables are evaluated from出evalue of stepsize

applied along山eprescribed search di陀 ctionfrom the current location of出edesign. The usage of GoldSection_nVar.m must accommodate function (5.17) and出e

need to deal with a design vector. Comparing with the single-variable case

U蝿 ge:

Value GoldSection_1Var('Example5_1' ，0.001，0，1，10) (5.14)

U皿 ge:

Value GoldSection_nVar('Example5_2'，0.001，x，s，0，1，10) (5.18)

Visit血巴 codeto s巴巴 thed巴凶15.

Compar/son w/th Sectlon 5.3: The code GoldSectlon_nVar.m was run from 山eCommand window using the fol1owing listing:

>>x [0 0 0 ]; >>S [0 0 6];

>> GoldSection_nVar('Example5_2'，0.001，x，s，0，1，10)

After 11 iterations the final Jine in the Command window was

ans CI

0.1000 1.2000

The above is interpreted as:

o o 0.5973

α1 = 0.1000; f(α1) = 1.2∞ x1 =0; A2=0.0;

which matches Section 5.3 and Equation (5.16).

x3 = 0.5973 (5.19)

5.4.2 Example 5.3-Two-Polnt Boundary Value Problem

A rea1 example for one-variable optimization can be found in出enumerical solution

of the laminar flow over a flat plate [5].百leproblem， usually attribut巳dto Blassius， represents an example of the exact solution of the formidable Navier-Stokes cquation. 百lemathematical fomlUlation， allowing for similarity solutions， is exp陀 ssedby a

third-order nonlinear ordinary differentia1 equation with boundary conditions

sp配 ifiedat two poi脳 -atwo-point boundary value problemσPBVP). What folIows is the essentia1 mathematical description.百leinte悶 tedreader can follow the details in most books on fluid mechanics， inc1uding出巴 suggestedrefe問 nce.


Mathematlcal Formulatlon

ff" + 2f" = 0

x=O; f(O)= 0; f(O)=O

x=∞ f(回 }=1

The solution， nondimensional velocity in the boundary layer， is obtained as

ず=f(x)

(5.20)

(5.21a)

(5.21b)

(5.22)

The solution to Equations (5.20) to (5.21) is largely through special iterative

techniqucs. These techniques use a numerical integration procedure li~巴出e

Runge-Kutta method to integrate the system by guessing and adjusting the missing

boundary conditions at the initia1 point such白紙 thefinal point boundary conditions are rea1ized. This is due to出efact that the integration procedu陀susually solve initial value problems only. In the Blassius problem，出iswould imply the missing initial value can be regardedぉ adesign variable.

('(0)=α

The objective function therefore will be

(5.23)

Minimize: ザ(∞)一 1)2 (5.24)

Implied in the formulation is伽 lf(∞)is obtained by integrating the di他rential叫uations(5.20) (山ωecan be considered as differential cons釘'ain臼).百lereis one othcr

consideration to take into account.百leintegration methods require the戸ublemto bc

expressed in state spa目 form，which r珂凶問叩 n山-ordぽ differentia1珂uation10 be expre鎚edas a system of n白百l-ordcr叫uations.百leconversion is fairly standard and is done through u溜lSformationand inu叫 ucingadditional v組曲les.For this example:

y，=f y~ =f=Y2

Y;=f' =Y3

Y;=f" =ず'/2

The optimization can be formulatedぉ:Find α

Minimize: [Y2(∞)ー lf

using the Runge-Kutta method (or叩 yother method) to integrale

(5.25)

222 NUMERICAL TECHNIQUE8-THE ONE.DIMENSIONAL PROBLEM

-EEEEEEEE『EEE」

噌

3vd

令

，

.

帽

3

、EJ

UJUJgJI

AU

ri--Ir --

JI11111J

r仰い同』

可EEEEEEaEEEEEJ

ooα

rEE

・E

・-E

・E

・-EL--

1BEEt-EEEBEEEEJ

)

)

)

nvnU4ηvi

(

(

(

-a

弔'-，，

fwlwlv

(5.26)

From published work [5J the value ofα・=0.33206. Inslead of出e白nalpoint at∞it is considered 10 be 5.0. Leaving白isvalue as a variable can define a corresponding two-variable optimization problem. In山eimplementation to follow，出epower叩 dconvenience of MA1LAB ar芭readilyapparent

Example5_3.m: This m-file is lIsed to implement expression (5.25). It calls the MATLAB built-in function ode45 10 use the Runge・Kutta4/5 order method. The function ode45 requires the state equations to be available in another m-file. It a1so E叫uires出edefinition of the interval of integration as well as the initial condition. The following line from the cod巴dcscribesthe usage (曲ecode is welI commented too)

[t y ] ode45('Ex5 3 state' ，tintval，bcinit);

t is the independenl variable and y the dependent vector.

Ex5_3_state.m: The state equations in (5.26) are available in出isfile.

stst [ y (2) ー0.5*y(1)*y(3)]'; ， y(3)

Refer to lhe code for details. GoldSectlon_1 Var.m is used from the command line for白isproblem as

U岨 ge: GoldSection_1Var('Example5_3' ，0.001，0，0.1，10)

Start iteration:

start alphal (low)

o 1.0000

alpha(工)

0.1528 0.1904

alpha(2) 0.2472 0.0378

The final iteration:

iteration 12 alphal (low)

0.3359

0.0000

alpha(l)

0.3364

0.0000

alpha(2)

0.3367

0.0000

alpha(up) 0.4000 0.0161

alpha(up)

0.3371

0.0000

5.4 ADDITlONAL EXAMPLES 223

The result expected isα= 0.33206. However， the tolerance specified was only 0.001. The initial bounding interval was only 0.4. The golden section has produced an impressive solution to山eTPBVP. Also note the convenience of modular pl'Ogrumming. No changes we陀 madeto UpperBound_1Var.m or GoldSectlon_1 Var.m to run this exarnple.

5.4.3 Example 5.4-Root Findlng with Golden Sectlon

The application of the golden section method to the root finding problem is important to accommodate cquality and inequality constraints. Consider Example 5.4:

g(α):0.75α，2 _ 1.5αー1=0

0$α$4

(5.27)

(5.2)

明 isis白einequality cons回 intof Exarnple 5.la except伽 tit is transformed to an equality constmint. In血isform theαnecessary to make it into an active constrainl can be established. The simplest way to accomplish this is to convert the problem to a minimization problem for whi抽出enumerical techniques have already bcen cstablished. Squaring Equation (5.27) ensurcs that the minimum value would be zero (if it exists). Therefore the solution 10 Equation (5.27) is the same as

M出mize:月α):[0.75a2 -1.5α-1]2 (5.28)

which can be handled by GoldSection_1Var.m. While山iswas quick and painless therear官 someattendant problems using this approach.目隠し thenonlin回 rityhas increased. This is usually not encouraged in numerical invcstigations. Second， the number of solutions has satisfying FOC increased. Figure 5.5 rep陀sentsthe functions in Equations (5.27) and (5.28).

'司

Solutlon UslngGoldSection_1Var.m: The following discussion is based on the experience of attempting to run thc golden section method with the lower bound at the value of 0，白ecase with出eothcr examples. The問 isa problem since the solution cannot be found. The problem app回 rsto be in the ca1culation of the upper bound. ln many of the numerical techniques there is an assumption白atthe objective function being dea1t with is山首modal，出atis， has a single minimum in the region of iaterest. For this to happen. it is assumed白at血efunction being minimized wi1l always decrease from the lower bound. Take a look at山efunctionfin Figure 5.5. It is increasing at thc va1ue of zero. UpperBound_1Var.m assumes that白einitia1 interva1 is too large and continues to bisect it to find a 釦nctionva1ue白紙isless曲anthe one at the lowcr bound. This drives the intervaI to zero and the function is exited with the lower bound and with no solution.

" 4

Remedy 1: The easiest way out of the difficulty is to changc the value of the lower bound and try again. It is useful to undcrstand that山巴problemis with the calculation

224 NUMERICAL TECHNIQUE5-THE ONE-DIMENSIONAL PROBLEM

18

-E---e--a・・'a--a・・・・''oa---

--a--v

J-Mhe-e

，

.••

16

14

12

10 (δ}て(HM)旬

ゐ

ふ

，

ふ

s'p-t' --e

4 solutlon

2

g

0.5 2.5 3 3.5

Flgure 5.5 Exampl自5.4.

of the upper bound (bracketing the minimum) ra山erthan with出巴 goldensection

method. Changing the lower bound to 1 and rerunning th巴 goldensection me出od

provides a successful 陀 sultwhich matches出egraphical solution. While success was realized. such a pr田 edureneeds user intcrvention ωwell as some knowledge about th巴problcm-inthis case there was出egraphical solution_ These are not acceptable qualitiωfor an automatic proc低lure.

Remedy 2: Sioce the positive slope at the lower bouod is indicative of the problem. it should be possible to sense this positive sign 01 the slope and move the

lower bound to the point where the slope is negative. Such a point can be discovered by the sam巴scanningprocedure. Thc slope computation is numerical

and is based on forward first difference. Oncc morc出ischange in the lower bound

is only needed for the uppcr bound calculation. UpperBound_m1Var.m is the

modified upper bound calculation which relocates the low~r bound to the point

where the slope is negative. The usage and information from running the

GoldSectlon_1 Var.m incorporating the chaoged upper bound calculatioo are

shown below. The samc usage with thc old UpperBound_1Var.m does oot produce any useful rcsults.

4

PROBLEMS 225

U岨 ge:GoldSection_1Var(' Example5_4' ，0.001，0， .1，20)

start alphal(low) alpha(l) alpha(2) alpha(up)

。 0.9931 1. 6069 2.6000

1.0000 3.0624 2.1720 0.0289

iteration 16 alphal (low) alpha(l) alpha(2) alpha(up)

2.5266 2.5270 2.5273 2.5278

0.0000 0.0000 0.0000 0.0000

REFERENCES

1. Burden， R. L.， and Faires， J. D.， Numerical Analysis. 4th ed.， PWS_KENT Publishing

Company， Boston， 1989. 2. Hosleller， G・H..samina-M.s..andMontaIOU-p.D.，Analytical，Nwnericd，仰d

computational Methodshrscience and Engineering，prmticc-Hall-Enslewood Cliffs.

NJ.1991. 3. Vー帥1酬・ G.N.. Nu抑制 Oplil仰 11011Teclmiq附 forEngineering Desigll，

McGraw-Hill， New York. 1984. 4. Huntlcy. H. E.， The DlvIlle Proporlion: A Sludy in Muthemalir:al Beauty. Dover

Publicauons， New York. 1970. 5. Schlichung. H.， Boundaη-Layer Theory， McGraw-HiU. Ncw Yo氏 1979.

PROBLEMS

5.1 Extend Example 5.1 to includc two inequality constraints for which a solution

exists-Display白 fωu鵬l(

5.2 Extend Example 5丘引.1to incIude two i加n叫ua叫alityconstraints for which t由herl巴1凶sno

solution. Display the function graphica11y.

5.3 Reprogram白ecode Sec5_2_2.mωobtain出evalues of f，中， and ct' through

MA'τ1.AB function programs回 dapply to Example 5.1.

5・4 間22J;;二13記念品222:l:::Z347tユ設計t

5.5 Identify the value of αwhere山econstr泊nt(5.3) b巴comesactive using由巳

Ncwton-Raphsoo technique.

5.6 Implemcnt a bisection procedure for the rninimization of山efunction of onc

variable. Apply it to solve Example 5.1.

5.7 For the polynornia1 apprOldmation implementation， cxtend出ccode to display

出eoriginal func凶on加 d出eapproximating polynomial.

5.8 For出cgolden section implcmentation， extend the codc to display the original function and thc anal external bounds on the variable.

226 NUMERICAL TECHNIQUE5-THE ONE.DIMENSIONAL PROBLEM

5.9 Veriちr出at白ebounds on the variable are decreasing by曲esame ratio. Why is this better白anthe bisection method?

5.10 How would you set up the golden section method to det巴rminethe zero of出巴

function instead of the minimum.

5.11 Combine出egolden section叩 d由epolynomial approximation to find the mmimumof出efunction.

5.12 Improv巴theaccuracy of the solution to the BI蹴 ius叫蜘1over that presented in the text.

5.13 Obtain the solution to

approximation method. the Blassius 巴quatlOn usmg the polynomial

6 NUMERICAL TECHNIQUES

FOR UNCONSTRAINED OPTIMIZATION

This chapter iIlus住atesmany numerica1 techniques for muJtivariable unconstrained optimization. While unconstrained op出血ationis not a common occurrenc巴 inengineering design， nevertheless the numerica1 techniques incIuded here demonstrate interes駐屯ideasand also capture some of the early intensive work in出earea of design optimi四 tion[1]. They aIso provide the means to solve constrained problems after they have been transformed into an unconstrained one (indirect m巴出odsー Chapter7).

This book is unique for the presentation of nongradient techniques (Section 6.2)

which can be empowered by the ubiquitous availability of incr芭dibledesktop

computing power. Until very recently. application of design optimization required mainframe computers that relied on large programs written in FORTRAN. Today's desktop can run programs白atreq凶reonly Iimited programming skills. The desire for globaI optimization has brought into focus numericaI techniques也atare largely heuristic， have limited need for sophisticated gradient-based methods， require limited programming skiHs. and require limited programming resources. Usually such techniques require白紙 thesolution space be scanned using a large number of iterations. These methods can ideally be run on personaI desktops for unlimited time. The pressぼ巴 oflimited computing resources is no longer a significant constraint on these techniques. While they may compromise on elegance and sophistication， it is important to recognize出巴 opportunityto engage desktop resources for greater

usefulness.

6.1 PROBLEM DEF1NITION

The uncons町ainedoptimization problem requires only the objective function. The book has chosen to emphasize an accompanying set of side constraints to restrict出巴

227

228 NUMERICAl TECHNIQUES FOA UNCONSTRAINED OPTIMIZATION

solution to祖 acceptableぬ i伊叩acel隠 gion.Most numerical附加iqu白 in出S

chaptEr agpo白血e副?∞ns回 intsin I叩 ngout恥 method.It i包su削s釦u剖d拘I勿y白恥e

附 P仰on附s討ib凶i日Ji町ty0ぱft恥h恥1砲ed由e叩町tωove町r的 t恥h恥e討副d也cco匂叩n附lIIntsas part of his exploration of出巴optimum.百ischecking of白cside∞ns甘副ntcan be easily programmed in the code. Thus， the problem for出ischapter can be defined as

Minimize f(X); 医]"

Subjectto: xf $; XI $; xr; i = 1，2， • • • ， n

6.1.1 Examp'e 6.1

;;2312::zz;222Z2;2JZem巴rpossible押 hicaJdescriptior

Minimize f(X) =f(xl・X2)= 3 + (XI -1.5x2l2 + (X2 _ 2)2

Subject to: OSX( S5; OSx2S5 (6.4)

Figure 6.1 provides the contours of出eproblem and出巴 e)lution can be located at

fJぷiZL2;arvalueisf=3百叩蜘nふadratic吋 hesolution

6.1.2 Necessary and Sufflclent Condltlons

2;江trzz;::2jr2zτs忠之江;tt:!sZ立;:?足立conditions) are白創出egradlents must vanish at the solution

21=2(Xt-15ら)=0

ま叩糾1句)+ 2(X2 -2) = 0

(6.5a)

(6.5b)

Equ叫ons(6.5) c叩beωvedto obtain xj = 3・xi= 2. This matches出egraph凶

器2iljitM223品官官民法iC別 :ttぉ;sz

問=ドニl (6.6)

(6.1)

(6.2)

(6.3)

6.1 PAOBLEM OE円NITlON 229

』

'j引J刊川川目』円阿川巾TIl!ll冶'lli

・，ι唯44whHM山川川印川仰l'I削

ta--

0.5 2.5 X1

5 3 1.5 2 3.5 4 4.5

Flguro 6.1 Example 6.1.

The Hessian is positive definite considering the deten凶nantsalone. Sec6_1_1.m' provides血E∞defor the symbolic calculation and also displays白巴g百phi回 ld白criptionωthe problem. The results of白ecalculation arc displayed in the command window.百lesatisfaction of the SOC identifies血脱出巴solutionfor the design is a minimum.

6.1.3 Elements of a Numerlcal Technlque

τ'he elements of a typi回 1num師団1techniquc or algorithm， can bc associated with the generic a匂ori伽nin位。ducedduring血ediscussion of the relevance of the one-dimensiona1 op白nizationfor multivariable problems (Chapt町 5).唄usalgorithm is iterative and is also referred to as a search algoritlvn， as the iterations take place by moving along a search direction or a search vectorduring an iteration. Thωes国民hdirections can bedetem山.edin many ways. The different臨 hniques白atare p隠sentedin this chapter primarily differ泊howthc search dir民 tionisωtablished.百le問mainingelements of the algori血m，ex伺 ptfor血巴convergenccJstoppingcriteria， are the same for almost all of the methods. The aJgorithm can be exprl凶 scdas follows.

IFileslO be downlonded rrom Ihc web si胞 nreindicaled by boldrace sons serir Iype.

230 NUMERICAl TECHNIOUES FO円UNCONSTRAINEOOPTIMIZATION

General Algorithm伺aηStep 1. Ch∞se Xo

Stcp 2. For each iteration i

Determine search direction vector 8j

(lt would be nice if the objective decreased along this direction)

Stcp 3. Calculate sX， =αsi

Note: aXj is now a function of the scaIar <<j as S， is known from Step 2

的 iscal1ed出estep size as itωtablishes出elength ofAXj

αj is d巴terminedby Minimizingf(丸山 whcreX'+I = Xj +企X1

Here， the convergcnce criteria (FOC/SOC)， and the stopping critcria (Is

design changing? Exceeding iteration count? etc.) must be checked.

i←i + 1; OoTo Step 2

ln this chapter， this algorithmic structure will be uscd for aIl of the methods.τ'he one-dimensional optimization wil1 be implemented using the golden section metbod. Example 6.1 wil¥ be used to dcvelop thc differcnt numerical methods.

6.2 NUMERICAL TECHNIQUEs-NONGRAOIENT METHOOS

τ'hese methods are aJso caJled zero-order methods.τ'he order refcrs to山eordcr of thc derivative of the objcctive function， necded to establish the search direction during any iteration. Zero order signifies that no derivatives制官 used[2]， which in tum implies白紙 onlyfunction values are used to巴S泊blishtbe search vector. There is another important significance that accompanies tbe missing derivative

computation-the FOC should not be applied in these methods. If wc were computing derivativcs we might as well inc¥ude this information for establishing the search vector. Only the changes in白eobjectivc function or the design variables can provid巴convergence四d/orstopping criteria.

羽田etcchniques are included in this section. The first is a heuristic one based on random search directions. The sccond one cycles through a search dircction based on each varlablc.百lelast is Powcll's method which has tbc property of quadratic convergence.

6.2.1 Random Walk

The search direction during each iteration is a random direction. Random numbers and

sets of random numbers are usual勿avai1ablethrough software. MATLAB includes the ab凶ty to generate severaJ typcs of random numbers and matrices. Most

computer-generated random numbers ar唱calledpseudorandom numbers as they cycle after a suffjcient number of them hav巴beencreated. The one吋dimensionalstcpsize computation is donc by thc GoldSectlon_nVar.m向nction(Chapter 5).

6，2 NUMERICAI. TECHNIOUEら-NONGAAOIENTMETHOOS 231

マHil』

A句orlthm:Random Walk (A6.2) 、Step 1. Choose Xo・N(number of iteratlons)

Set i = 1

Step 2. For each iteration I

81 = Random vector Step 3. Xj+1 = Xj +α，Sj

α'j is determined by minimizingj{Xi+I)

i←ー i+ 1

lf i < N 00 To Step 2

elsc SωP (iωrations exceeded)

EZ2222zteおszz:;12rz::;;むれは也事i

圃l;説話器忠告部諜誌記f悲:;訪問sign based on a test of random number.

RandomWalk・m:Tlmrandomwalk algorithm tA62)is impiementcd in this

OP6miadon-TMcodeof the random walk function also includes thE following

1'1

I~唱

，~・t

出econtour of thc objective is drawn. • For two-variable problems tn • Fortwo回variablEproblemsthe successful iterations Me tracked on the contour

d the iteration" number identified.

.2;:u16vm油附d…“…削仲間10的 C叩刊記dfor

the one-dimensional stepsize computation. . The multivariable up戸rboundcalculation (developed in chaptcr5)is used for

. ;口?コ江::;E:泣:誌広:2:zr1;z;;ニ11iIn1 山 here 出伽0…即凶帥叫川hd吋di曲枇iI1泊附ωr肉悶e回c

.:エ::二i: :エ;:ニeL制 ignvariables and山 ctionsfor all of

thc itcrations arc printed.

232 NUMERICAl TECHNIOUES FOR UNCONSTRAlNED opnMIZATION

Use of previous code iIIus町atescode reuse and modular programming. Once more r伺 dersar宅 stronglyencouraged 10 visit the code to understand th巴位anslationof the aJgorilJlm and subsequent application.

U掴 ge:司

Z RandomWalk('Example6_1'， [0.5 0.5] ，20，0.001，0，1，20) (6.7) .~

事

Input: (They are th巴itcmsenclosed within the outermost parenthesis in expression (6.7).) a

賞'Example6_1・:[0.50.5] 20 0.001

0

20

百lCfile Example6_1.m where lJle objective is described starting dcsign veclor (dvarO) number of iterations of the method (niter) tolerance for出egolden section proced町 e(tol) initial stepsize for scanning upper bound (Iowbound) slep interval for scanning (intvI) number of scanning steps (n肘aIs)

Output:

A vector of n + 1 vaJues of design variables at the last iteration and the vaJue of the objectivc function.

Executing function (6.7) in lJle Command window produces the following information (出ecomments are added later):

-The design vector and function during the iterations

0

0

1

1

1

1

2

2

8

8

4

4

4

4

4

4

0

0

9

9

9

9

8

8

0

3

0

0

0

0

0

0

内

ununununununヨnヨヨ3nEnononunDnD内

回

=RJRd守

，

守

，

司

，

守

'ιuζu日Jnunvnununununu

....•••••

4

・・・・・・・・・・

m

0

0

0

0

0

0

0

0

2

3

3

3

3

3

3

3

a 0.5000

0.5000 1.0440 1.0440 1.0440 1.0440 0.9756 0.9756 2.1093 2.0481 2.0350 2.0350 2.0350 2.0350 2.0350 2.0350

5.3125 5.3125 4.6481 4.6481 4.6481 4.6481 4.6349 4.6349 3.0663 3.0025 3.0020 3.0020 3.0020 3.0020 3.0020 3.0020

告 start

者 didnot move も decreasedfunction 告 didnot move 告 didnot move 告 didnot move 告 decreasedfunction 告 etc

6.2 NUMERICAL TECHNIOUES-NONGRADIENT METHODS 233

3.0804 3.0804 3.0804 3.0804

ans 3.0804

2.0350 2.0350

2.0350 2.0350

3.0020 3.0020

3.0020 3.0020 も 20iterations

2.0350 3.0020

Compare自問 vaIuωto山eactual叫 utionof xi=3，xi=2，j*=3. T出sis remark且hlefor a method出athas no sense of preferred direction. If the execution 15 resated with Menucalinput，m entimnew solution will beobtained.Figure62tracks th;伽 Igesofthe d叫 nvari枇 son the co脚 lfplot. The movem巴ntof ilie血sign

variables are paired with the corzesponding iteration number.Therandom walk function can bEExECUted for several diffErent starting sets of design variables.It should not disappoint. The GoldSection_nVar.m is山esarne one as in Chapler 5

except白紙ilieprinting has been suppressed. While executing出eRandomWalk.m the printing of山 iteration山mb?

cmates a mess as you approach the solution.You can switch it offby commenttqgme 旬以(ー・)statement-ThgeodeinRandomwalk-m also seIves as a templateforother

algoriilims. lt is worth exploring in detail.

k

0.5 1.5

Flgure 6.2 Random walk: Example 6.1.

ijLili--

1

1

J

i

l

l

ー

234 NUMEAICAl TECHNIOUES FO円UNCONSTAAlNEDOPTIMIZA TION

6.2.2 Pattern Search

The Pattem Search method is a minor modification 10 lhe Utlivariate method with a

maJor Impact. As the Pattem Search uses the Univariate method the latter is not discussed separately. In the Univariate method， a1so known悩 theCyclic Coordinale Descenl method， each design variable (considered a coordina包)provides a search

direction. This is a1so refe汀edto as a coordinate direction，四dis easily expressed through t恥h恥1聡eu叩n山1

n削umb邑釘roぱfv副aria油叫blesωs山d也e鈴側s討ig伊nvariables) i氏teratio叩nsonefor 巴悶ac凶hoft白hen search directions. It can be shown bv appIication伽 tfor problems with considerable nonlinea均， theUnivadatemet130J tends to get lock叫 intoaz討igz却agp仰at悦t胞E叩emofs叩ma】m叫a由巴s叩olu山Jt山110叩n.

τllePat旬mSearchprc問 dureattempts to disrupt this zigzag bebavior by executing one additional iteration for each cycle. In each cycle， at白eend of 11 Univariate directionsーthen+lsea陀hdirection is assembl伐1as a I伽II加n即脚1隠巴悶即arc∞伽oω叩m】11蜘b

d恥伽町附c伽“on悶s聞削d曲恥eoptimum 叫 u凶eof t山h恥le巴蹴 p阿討悶zefo叫 atdirection. 正one-dim叩 ionaloptimal stepsize is白en∞mputed佃 dthe next cycle of iteration begIl1s.

A勿orithm:Pattern Search伺6.3)Step 1. Choose X1， Nc (number of cycIes)

j(1) =j(X，); Xc(1) = X， E，・ E2:tolerance for stopping criteria

Setj = I (initialize cycle count)

Step 2. For each cycle j

Fori= 1，11

S;=会(univariate山 p)

X;+， =X;+αβ， α/ is determined by minimizingj(X;+，)

(store values ofα.i and S/)

cnd of For loop

n

号=玄α:;S;aX，肺， -X， (Pattcm取肘

Xj=Xn+， + CちiSj

X心・+1)←Xs J"心・+1) =j(再)(stor巴cyclevalues)

Step3. 6j=1rU+ 1)-1rωAX=X，.u+ I)-Xcω If l.1jl S E" stop

IfAXT .1x S ~. stop Ifj=N"， stop

X，←~

6.2 NUMEAICAl TECHNIOUES-判ONGAAOIENTMETHODS 235

j ←j+l

GoToStep 2

ThePatternSearchmethodhasafcwadditionalstoppingcritcria.Thcscarebasedon

how much thefunction isdecreasing for each cycleand how much change is taking placein thcdesign var iableitself. For thelatter thisIl1formation isbased on thelength

ofthechangein thedesign vector for thecycle. Onceagain theapproach issimpleand the implementation is direct. Programming such a technique will not bc too forbidding.

PatternSearch.m: To conlInue出eexposure 10 MATu.B programming出ismethod

is implemented very different1y from the other algorithms that have bcen translated into code so fi訂 .It is a stand-a1one program. Therc are prompts for山 erinputs wi山

appropriate default values in cas巴 theuser decides not to enter a value. The

one-dimensional stepsize computation is implemented by血emultivariable golden section mcthod. The interesting featu問 sare as fol1ows.

. It is a freestanding program.

・百lefile contaIl1ing the objective function is selected through a list box.

• Scveral program control par宙netersare entered from the keyboard after a suitable prompt indicating defllult values.

• Dcfault values arc initialized if the user chooscs not 10 entcr the values.

• Thc design vcctor is tracked and printcd aftcr thc program is stopped.

• Since the upper bound calculations arc based on positivc valucs of stepsize， thc search direction is陀 vcrs吋 if由巳問tumedstepsize is出esame as the lowcr

bound.

• Iteration counts are recorded.

• For two-variable problem the objective function contours are plotted.

. The first nine or Icss iterations are traced on出巴contourplot.Thc Univariateand the pattem steps are co)or codcd.

U掴 ge: PatternSearch (6.8)

The prob)em is Example 6.1.百lefi]e is Example 6_1.m.

Output (MATl.AB Command window): With山edefault va)ues and the starting vector [4 3] the following are written to the Command window.

>> PatternSearch The function for which the minimum is sought must be a

MATwa function M -File. Given a vector dependent variable it must return a scalar value. This is the function to be

MINIMIZED. Please select function name in the dialog box and hit return:

噌司

236 NUMEAICAL TECHNIOUES FOA UNCONSTAAINED OPTIMIZATION

The function you have chosen is ::Example6_1 maximum number of cycles [1000]:

convergence tolerance for difference in f[le・8]:

convergence tolerance on change in design x [le-8]:

Input the starting design vector. This is mandatorv as

there is no defaul t vector setup. The length ofシourvector indicates the number of unknowns. Please enter it now and hit return:

[4 3]

The initial design vector [4.00 3.00]

Convergence in f: 3.999E嗣 009reached in 10 iterations

Number of useful calls to the Golden Section Search Method: 31

Total number of calls to the Golden Section Search Method: 40

The values for x and f are

4.0000

4.4992 4.4992 4.3868 4.1432

4.1432

3.8229 3.3338

3.3338

3.1750

3.1980 3.1980

3.1944 3.1473

3.1473 3.0346 2.9958 2.9958 2.9971

2.9972 2.9972

2.9972 2.9977

2.9977

2.9999

2.9998

3.0000 3.0000

2.6924 2.7616 2.7616

2.5287 2.2224

2.2224

2.1543

2.1321 2.1321 2.0916

2.0980 2.0980 2.0682 1. 9969 1.9969 1.9981

1. 9981 1. 9981 1.9986

1.9988

1.9988

1.9990 1.9999

1. 9999

4.2500

4.0000 3.6916 3.6398

3.5801 3.4022 3.2890

3.0494

3.0343

3.0180 3.0175

3.0121 3.0118 3.0096 3.0067

3.0016 3.0000

3.0000 3.0000

3.0000 3.0000

3.0000

3.0000

3.0000

3.0000

3.0000

0

0.4996 0.3081 0.2254 0.2446 0.2339

1. 3148 0.4896

0.0681

0.3248

0.0234 0.0405 0.1579 0.0471 0.0298 2.3916 0.0391 0.0013 0.0322

0.0001

0.0005

0.4563

0.0005

0.0002 4.9305

0.0002

6.2 NUMEAICAL TECHNIOUE9-NONGAADIENT METHODS 237

2.9998

2.9998 2.9998

2.9998

2.9998

1.9999

1.9999

1.9999

1.9999 1.9999

i

t

i

l

l

-

-

0.0000

0.3398

0.0000 0.0000

0.2020

3.0000

3.0000

3.0000

3.0000 3.0000

To verify白紙出eprogram is executing the a1gorithm， sc佃 thedesign veclors岡山E

first lWo columns. The univariale steps can b巴recognizedwh巴none component of the vec10r changes while由巳 othermaintains its value. The pattern directions can be

;1. recognized when both components change.

~: The number of iterations is a significant amount. One reason is that the tolerances are much smaller. It can be noted that由巳 differencebelween lwo ilerations ch叩 gesIinearly. Such methods are expec1ed to run for a large number of iterations. The

，1: program con佐01par羽田terscontrol the acc町冨cyof出eresults as well as由enumber of iterations. The solution is more accurat巴thanthe Random Wa1k me出od.

Figure 6.3 iIIustrales出ea1gorithm graphically. The univariate and the paUern directions are quite distinguishable. When executed on your machine th邸巳 stepsare explicitly color coded. Readers are encouraged 10 go over the code. and modify i1 to

E

4.5

0.5 1.5 3

Rgure 6.3 Pallem search: Example 6.1.

238 NUMEAICAL TECHNIQUES FOA UNCONSTAAINED OPTIMIZATION

refl配 tth巴irpreference. The number of lines is around 250. We at琶湖rtingto get into

senous programmmg.

6.2.3 Powell's Me'肋od

lf there were only one zero-order method白紙 mustbe progranuned， the overwhelming choice would be Powel1's method [3]. The principal reason for曲edecision would be that it has the property of qlladratic convergence， namely， for a quadratic problem with n variables convergenc巴willbe achieved in less than or equal to n Powel1 cycIes. A quadratic problem is an uncons住温inedminimization of a

function that is expressed as a ql.ωdratic polynomial-a polynomial with no term having a degree greater than two. Example 6.1 is an example of a quadratic

polynomial in two variables. Engineering design optimization problems are rョrelydescribed by a quadratic polynomial.百世sdoes not impl y that you cannot use Powel1' s

method. What this means is that the solution should not be expected to converge

quadratically. For nonquadratic problems， as the solution is approached iteratively，出eobjective can be approximated very well by a quadratic function. It is at山isstage 出atthequadr副cconvergenc巴propertyis reaJized in the computations.

The actual aJgorithm (A6.4) is a simple (the word is not being used Iightly) modification to the Pattem Search algorithm. ln each cycJe (a仇er出efirst)， instead of using univariate directions in the first n iterations. the search directions are obtained from the previous cycJe. The new search directions are obtained by Iψ shifting the directions of the previous cycJe. In出isway a history of the previous search directions is used to establish the next. With this change. Powell's method should converge at most in three cycJes for Examplc 6.1 rather than the several

listed in the previous section.

A匂orithm:Powell's Method仰6.4)

Step 1. Choose X" Nc (number of cycles)

Jtl) = j{X1); Xc{l) = X， E，・ E2:tolerance for stopping criteria

Setj = 1 (initialize Powell cycle count)

Fori= 1， n

S; =~; (univariate step) Step 2. For each .cycle j

Fori=l.n

If j注2，S;←S;+，X;+， =X;+αISI

α; is detennined by minimizingJtXi+')

end of For loop

Sj=S;+， =Iα，;S; = Xn+， -XJ (pattem step)

62 NUMEAICAL TECHNIQUEら斗唱ONGRADIENTMETHODS 239

「11川川イ川引叩1

f

H

{

Xf=Xn+， +α戸j

XcU+ 1)←Xf; fcU+恥 f(Xj)(s附 cycIe叫附

Step 3. !:if= fcU + 1)一五，U);d1{ = XcU + 1) -Xc(j)

lf ldfl S E" stop

lf d1{T dX S E2. stop

If j=Nc• sωp

X，←XJ

j ←j+l

GoToStep2

APPMeatton of poweII's Method:The translation of the algorithm into MATLAB

COdeisleft as m exercise for the student.Step-by-step applicaGon toExample 6.1 is

shown here. Example 6.1 is res旬飽das

Example6.1

Minimize f(X) = f(x" X2) = 3 + (x， -1.5X2)2 + (x2ーが (6.3)

ー

X， =[g:~lfl仲5.3125; E， =

S， =[6]; S2 =[~]

(6.4)

j=l

Step 2. j= 1

X2431+αl国=??l

2. 吋=1/4; X2=防l

X3 =問+句砕[o.~ユ1444615;X4zj;沢民)=4.5576

S) = 0.25 [6] +仰輔自)[~]=[ぷ5]X4=[03251ベぷi5]

i = 2:

i=3:

241 6.3 NUMERICAL TECHNIQUEs-GRADIENT.BASED METHODS

ilillit---

In light of the previous definition. these would be referred to asfirst.order m巴出ods.The search directions will be constructed using the gradient of the objective function. Since gradients are being computed.出eKuhn-Tucker conditions (FOC) for unconst凶 nedproblems. Vf = O. can be used to check for convergence. The SOC are hardly ever applied. One of the reasons is血atit would involve the computation of an n X 11 second derivative matrix which is consid巴redcompuぬtionallyexpensive. particularly if出eevaluation of the 0何回tivefunction問quiresa call to a finite element method for generating requir叫 informalion.Another reason for not calculating the Hessian is血atthe exislence of白esecond derivative in a real design problem is not certain even though it is computationally possible or feasible. For problems that can be described by symbolic calculations. MA11.AB should be able to handle computation of second derivativ巴atthe possible solution and i岱 eigenvalues.

Without SOC these methods require user's vigilance to ensure白紙thesolution obtained is a minimum rather山ana maximum or a saddle point. A simpl巴way10 verify出isis to perturb the obj巴ctivefunction through pe目urbationin th巴designvariables at the solution and verify it is a local minimum. This brings up an important prope町 oftheseme血ods-theyon1y find local optimums. Usually this will be c10se to th巴 designwhere the iterations are begun. Before concluding the design exploration. it is necess釘 Yto execute the method from several starting points to discover if other minimums巴xistand select the best one by head to head comparison. The bulk of existing unconstrained and constrained optimization methods belong to出iscategory.

Four mcthods are presented. The first is thc Steepest Descent method. Whil巴thismethod is not used in practice. it provides an excellent example for understanding thc algorithmic principles for臨 gradient-basedtechniques. The second is the Conjugate Gradient technique which is a classical workhorse pa凶cularlyin indus町 usage.The third and fourth belong to the category of Variable Metric methods. or Quasi-Newton me出odsωtheyare also called. These methods havc been pop叫釘forsome time.回 dwill possibly stay that way for a long time to come.

Thc gcneral problem and specific example ar志向producedfor convenience

NUMERICAL TECHNIQUEs-GRADIENT-BASED METHODS 6.3

240 NUMERICAL TECHNIQUES FOR UNCONSTRAINED OPTlMIZATlON

L=11:570; f(X4) =4.4843

α・=-0.14111'¥'Y 川.855811- 吋..l~OO; 1¥.2 = 11.01041; f(X2) =4.4145

α; =0.4235;

Step 3. l.dfl = 1.353> EI' continue

ldXT dXl = 1.8992 > Ez. continue

} = 2: One cycle over

GoToStep2 Step 2:}= 2

xJ0.855りJ =11 1c'7n 1: 11.l570 l'

同

M--gu

二111 iZ] ; S島←叶吋2戸イ2=[0イ=[0ら0品2沼2払5J叫山=0ω附叩ω4伺仰側0的防3お叫5

i= 3:_ v _1 0.95671 ~ r 0.10ω11 r'll 吋 -U.1966 J ~3 =伊.0396/:α3=20以~= /2/; f(X4) = 3

おご抗出土::ニi;;32!emd恥 C叫 utationsin出i…ercise蹴 stopped

山itE:27忠;:Jr:atJ;;::::wouldit叫 here?You蹴巴…agedto

-ηle method only took two cycles to converge (as expected)・

・Theextent of compu凶 onis恥 sameas the Pattem S即 chb川 lenumberof iterations is significantly smal1er.

・2:;;212S22;お訟は:rsecondmleiS叩 tive.H恥E叩町町n即附c印e

• The stepsize for the last itcration is 20ぉ.The supporting algorithm for ;;:::!1gand golden secuon mustbe able tO凶 tpthe minimum a剖t伽 elar

• Whi】討ileno削otilh加u凶I犯sl回m即t鈴凶ε吋dh恥m町er凧E久.th白恥h恥elsa制x対iml剛3机加削u刷叩1m町nm叩Il川u叩um】be凶?引r印'0ぱfc句y戸c1eωstωoc∞onv刊er:屯ge伺n町1にc印efl白or日a z詑::抗r払L:忠:立::z2ぷ1沈均衣な;f:ユ:f曲目町gpoint. Y… (6.1)

(6.2)

(6.4)

(6.3)

j(X); [X]"

XI~Xj~xlt; i=I.2.....n

j(X) = f(XI. X2) = 3 + (XI -1.5x2)2 + (X2 -2)2

OSXI S5: o SX2S5

Minimize

Subjectω

Subject to:

Minimize 訪協2jz:32ifi:5132ti:誌記;容認matrix. lf sea帥 directions S.品ja佃n吋d号a町o叩n吋叩~u昭g伊at，除e di廿悶ec“伽oα∞n眠Ec∞on.町リ肝卯uga舵cya訂rc

Steepest Descent Method

This method provides a natural evolution for the gradient based techniques [4]. The gradient of a function at a point is the direction of the most rapid increase in

6.3.1 (6.9) SJ[U]Sj= 0

Veri今ifit is truc for thc example.

242 NUMERICAl TECHNIQUES FOR UNCONSTRAINED OPTIMIZATION 6.3 NUMERICAl TECHNIQUEs-GRADIENT-BASED METHODS 243

the value ofthefunctionat tllat point. The descentdir巴ctioncan be obtained reversinl!

the gradient (or multiplying it by -1). The next step would be to regard tl1e descent

veclor as a search direction， after all we are aUempting 10 decrease tl1e function

Output (written to白eCommand window): A couple of iterations at出estart and at

司自EEnd(out of20iterations)Me copied b-low.

through successive iterations. This series of steps give rise to白eSteepest Descent algorithm j A匂'orithm:Stl僧 P白 tDescent伺6.勾

Step J. Choose XJ， N (number ofiterations)

!.(l) = j{X1); X.(l) = X1 (store values)

EJ， E2' EJ: (tolerance for stopping criteria)

Set i = 1 (initia1ize iteration counter)

Step 2. Sj = -Vf(X/)(出isis computed in Step 3)

X;+l =X;+α~;

的 isdetermined by minimizingf(X;+I)

Xs(i + 1)←X件1; 五(i+ 1) = f(Xi+l) (store va1ues)

Step3. aJ=!.(i+l)ーぶりaX= Xs(i + 1) -Xs(i)

IflaJl ~EI; 陶p (function not changing)

If aXT aX ~ E2; stop (design not changing)

[fi + 1 = N; stop

If Vf(X;+llVf(Xi+l) ~ E3; converged

i←i+ 1

GoToStep2

SteepestDescent.m: This is an m-file白a1executω a1gorithm A6.5. It uses白e

golden section and tl1e upper bound scanning PI1凹ess.百leたaωresofthis program are as follows.

• For two variables it wiJI draw tl1e contour plot.

• For two variables the design vector changes can be seen graphically in slow motion witl1 steps in different color.

• The design variables，出efunction va1ue， and出eSqUar巴oftl1e lengtl1 of曲巴

gradient vector(ca11ed KTvalue) at each iteration aredisplayed in tl1e Command window at completion of tl1e number of iterations.

• The gradien1 of tl1e function is numerically comput叫 usingfirst forward fisIte difference. The gradicnt computation is tl1erefo陀 automatic.

• From a programming perspective， basic string handling to change line color is introcluced.

Usage: SteepestDescent('Example6_1'， (0.5 0.5]， 20，0.0001，0，1，20) (6.10)

I~

j

0.5000 0.5000 5.3125 5.2969

0.5996 0.9483 4.7832 2.8411

1.0701 0.8429 4.3766 3.1379

2.6901 1.8659 3.0298 0.0507

2.7455 1.8502 3.0233 0.0462

2.7585 1.8960 3.0181 0.0316

Figure 6.4 rep陀 sentsthe graphical rnotion of the design variables for 20 itErations.

The graphical picture and the changes in the design arE not imPEESSiVE・TheSteepest Descent mEthod iS WOEfully MadequatE Compazed to Powell-s method even ifthe latter

is a zero-order metl1od. MO[1回ver，白isconclusion is drawn with rωpect lo an回 syproblem (quadratic) Example 6.1.百isperformance justifies tl1e lack of serious

interest in the Stecpest Dcscent method.

5

4.5

0.5

円gure6.4 Sle邑pestdesωnt: Example 6.1.

244 NUMEAIC泊LTECHNIQUES FOA UNCONSTAAINED OPT1MIZATlON

rJぷ;なお?ね12:ロ:JI出:241詰誌な2EZ誌な巴xpectedto overcom巴thispattem. In the case of the Univariate m仙 odthis was

achEeyed through patFrn Search method in the zero-order family.An iteration

breaki時 0円ofthe zigzag pa旧 m (or preventing getting locked into oInEP、i5:芯:芯::rro:九ばliz:::尽忠:立ロ22:口eme叫e仙白o山d“Su叫ch川】

6.3.2 ConJugate Gradient (Fletcher-Reeves) Method

TheCo吋ugateGradient method， originalIy dueωF1etcher and Reeves [5]， is a smaIl

間宮ei器z器部ii;号帯?:諮;諮詳;5鵠諜鵠5謎誌器25g:;:混2訳;qi詫号告:?告:1i詰z能i5;鴇詰諮; c∞onv，叩erge叩nc巴b ecau se th e s ea rch d l Ir.e c tu1 o En3 s aMr巴cω01ベゆIjψjugatew川it出h陀s叩p民削ttωothe Hessian

:ご工幻:?xi:E:広:包二誌ば:ごナlu山u凶“ωo叩n.A qu聞帥la

A匂'or/l的m:ConJuga俗 Gradfj開 t伺a句Step 1. Choose X" N (number of iteration直)

!s(l) =j(X，); X.(l) = X，(store values)

E，・ E2・E3:(tolerance for stopping criteria)

Sel i = I (inilialize iteration counter)

Step2. Ifi= I，Sj=-Vj(Xj)

Elses= 可'(xjfVj(Xj)

Vj(Xj_l) T Vj(Xi_')

Si=-Vj(丸)+ sSj_， X'+I =X/+αβt

αI is determined by minimizingj(Xi+，)

X.(i + 1)←X;+，;五(i+1)=j(X川)% (sω陀 V山巴s)

S句 3.!J.j=fs(i+l)ーが1); 企X=X.(i+1ト X.(。Ifl!J.jls凶E町Iμ; 岬 (仇f仇向u町m附削Jn肱刷刷n恥削刷E陀削削附cω叫ti山ionn刷E

I旺f!J.X'!J.X:$E2必 stωop (design not changing)

Ifi+ I =N; 5tωop

IfVj(Xi+，fVj(X件，):$ E3; converg巴d

i~ i+ 1

GoTo Step 2

cふ;::P22Jt::2JUlz;ittrtt;123tzr:rn;ぷ;。;:;

6.3 NUMERICAL TECHN，QUEs-GRAD'ENT.BASED METHODS 245

computation--s. and th巴 succes5ive adjustment of 恥 search direction

incorporating this value. ~ represents the ratio of the square of the current gradient vector to the square of the previous gradient vector. The first thing to nOlice is山ata degree of robustness is built into the method by carrying information from the previous iteration. This is Iike maintaining a history of the method albeit for just one iteration. Since the FOC is based on the length of the gradient approaching zero at the solution.出isP訂 ticularfonn of incorporation of ~ is ingenious. If the previous ileration is close to the solution. then s is large and出巴previousiteration plays a significant role in the current iteration. On the other hand， if s is large. suggesting the solution is still far away， then the current value of the gradient determines the new search direction.

Appllcatlon 0' ConJugate Gradlent Method: This method is important to warrant working through the iterations. These caIculations， similar to Powell's m巴thod，can be done using a hand caIculator. using a sp陀 adsh回 t，or usingMATLAB itself. The author recommends the caIculator as白eprocess is slow enough to develop an intuition working with the paltem of numb巴rs.In the following・andunlike Powell's m巴thod，on1y the relevant caIcu1ations are recorded. The reader should follow aIoog verifying the numbers are iodeed correct.

10.51 Step 1. X1 =1 o:sl; f(X，) = 5.3125

1 0.5 1 • _. ___ .... . ___. __ 10.59961 Step 2. S， = 12~i51;αi = 0.1993; f(αi) = 4.7831; X2 =1 O:94841

t=2|-1.64591 Step 2: s = 0.5351 ;町民)=IO~36571

11.91351 牛 1312 = I O:83831;α，i = 1.2544; f(叫)=3.0; X3 =121

Con)ugateGradlent.m: This m-file will execute the Conjugate Gradient method. For two吋 ariableproblems there will be a contour plot over which出eiterative progress in the desi伊 spaceis recorded. The property of qua合8tiCconvergence can be observedぉ thesolution is obtained after two iterations. This is definitely impressive compared with出巴 St田 pestDescent m巴出od.K田 pin mind出atthe modification to the aIgorithm is minor. As出evarious codes are run in order. some changes in the figure can be observed. These are larg巴lydue to the use of some string processing functions. Y ou are encouraged to see how白eyare implemented. The author is certain there are出ingsyou would do differently. Do not hesitat巴to甘y，as 白紙isthe on1y way to learn to use MATLAB eff，巴ctively.

U蝿 ge:ConjugateGradient(‘Example6_1'， [0.5 0.5]， (6.11) 20，0.0001，0，1，20);

246 NUMERICAL TECHNIQUES FO円UNCONSTRAINEDOPTIMほATION

Output (Command window)

The problem: Example6_1

The design vector， function value and KT value during the iterations

0.5000 0.5996 2.9995

0.5000 0.9483

1.9982

5.3125 4.7832

3.0000

0.2490 2.8411

0.0001

Keep in mind the solution is dependent on the tolerance for the one-dimensionaI stepsize search and that the derivatives are computed nurnerically. The execution of 白ecode should display Figure 6.5 in the figure window.

6.3.3 Davldon-Fletcher-Powell Method

The Davidon-Fletcher-Powell (DFP) [6] method belongs to山efamily of Variable

Metric Methods (VMM).It was nrst introduced by Davidon and seVEral yEMB later

was developed in its current form by Fletcher剖 dPowelI. B巴ingaware of the

Conjugate Gradient method， these methods would not merIt incIusion in this book if

they did not have the property of qu.adratic convergence， which they do. Generally

2 3.5 4 4.5

Flgure 6.5 Coniugate gradlent method: Example 6.1.

屯川崎喝刈話冗一同詔一

5

6.3 NUMERICAL TECHNIQUEふ-GRADIENT・BASEDMETHODS 247

this far凶lyof methods go beyond that. As the solution is approached， they behave like Newton's method (Section 6.4). The quadratic convergence of Newton's method is

quile impressive-it will locate山 solutionto a quadratic problem in olle iteration.

This Newton-Iike behavior of the VMM， as the solution is approached， has given them anothernam巴termedQuasi-Newtoll or Newton-like methods.

The DFP method is presented here for historical interest血 dbecause it is a little

easier to understand than由巴others.In fact， Huang (7) documents a generic procedure

from which most of出epopular methods can be obtained， and from which you could also derive your own.

The Conjugate Gradient method' s irnprovement over the Steepesl Descent method was

possible because of the inclusion of the history針。m the previous iteration. ln the quasi-Newton methods th巴 historyfrom aIl previous iterations is available. This

inform岨onis collected in an n X n m縮 ixcalled血emetric. The metric is updated wi出

eacb iteration釦 dis used to establish the s回詑hdirection. An initial choice for the metric

is also 明 u田d.It must be aり刷tetバ'cpositive deftnite matrix. For the m抽 odto

converge， the metlIc must hold on to its positive definite property through the iterations.

h出eDFP method， the metric approaches由巳inverseof the Hessian at Ule solution.

A句orithm:Davidon-F，制 'cher-Powell(A6.刀Step 1. Choos巴x..[A1l (initial me甘ic}，N

Eh e2' eJ: (tolerance for stopping cri町 ia)

Set i = 1 (initializ巴iterationcounter)

Step 2. 8，=ー[Aj]Vf(Xj)

Xi+1 =Xi+α，su AX=αjS， αi is determined by minimizingf(Xi+l)

Step 3. If Vf(Xi+I)T Vf(Xi+1)三e3; converged

If !f(Xi+l)ーf(Xj)1::;; el; stop (function not changing)

IfAXT AX~e2・ stop (design not changing)

If i + 1 = N， stop (iteration Iimil)

Els巴

Y = Vf(Xj+l) -Vf(Xj}

z= [A;]Y

AXaXT

[B]=---:-::::;:::一AX'Y

ZZT [q= ー--~-

V'Z

[Ai+Jl = [A，] + [B] + [q

i←i + 1

GoTo Step 2

248 NUMERICAL TECHNIOUES FOR UNCONSTRAINED OPTIMIZATION

In the above白ematrices are enclosed by square brackets. The initial choice of the

metric is a positive definite matrix. The identity matrix is a safe choice.

DFP.m: The DFP algorithm (A6.7) is coded in山ism-file. Similar 10 the previous

programs， for two variables山edesign changes are tracked on a background contour

plot， Figure 6.6 . The default initial metric is Ihe identity matrix generated using a

MATLAB built-in function.

Usage: DFP(' Example6_1' ， [0.5 0.51，4，0.0001，0，1，20); (6.12)

Output: The output from the above invocation (copied from the Command window) can be obtained by removing thc scmi-colon after the statement. The variable

definitions match those defined in the algorithm. For Example 6.1

0

1

=

ーム

nU

A

4 4.5

FJgure 6.6 DFP Melhod: Example 6.1.

5

6.3 NUMERICAL TECHNIQUEふベ3RADIENT.BASEDMETHODS 249

iteration number: 1

s 0.4990 2.2467

delx = 0.0996 0.4483

y

-1.1458 2.6153

z -1.1458 2.6153

B

0.0094 0.0422 0.0422 0.1899

c 開 0.1610 0.3676 0.3676 -0.8390

A 0.8483 0.4097 0.4097 0.3509

iteration number: 2

s 1.2443 0.5446

The problem: Example6_1 The design vector， function value and KT value during the iteどations0.5000 0.5000 5.3125 5.2969 0.5996 0.9483 4.7832 2.8411 2.9996 1.9987 3.0000 0.0000

6.3.4 Broydon-Fletcher-GoldfartトShannoMethod

If you were to program only one gradient-bascd m巴thod.then thc Broydon-Fletcher-Goldfarb-Shanno(BFGS)181method would be the one.It is a quasi-Ncwton method and currently is the most popular of th巴 VariableMctric methods. It enjoys lhe propcrly of quudratic convcrgencc and has robustness by carrying forward information from I13E previous iterations.The differencEbEtween 由巳 DFPand BFGS is the way the melric is updaled. The former converges to the

inverse of the Hessian. while the latter converges to the Hessian itself. In a sense 山eBFGS is morc direct. The BFGS has replaced the Conjugate Gradienl

techniques as a workhorse in solving nonlinear equations.

寸Hjil--J41JHAHill-Jハー

250 NUMERICAL TECHNIQUES FOR UNCONSTRAINED OPTlMIZATlON

For convergence the metric must be positive definite. An initial choice of

positive definIl巴 matrixfor the metric is usualJy sufficienllo ensure this property for quadratic problems. The identity matrix is usually a default choice.

Algorithm: Bro附加・F倒的er-Goldfarb-Shanno(BFGS) Method (A6.却Step 1. Choωe X" [Atl (initial metric)， N

E" E2・E3:(tolerance for stopping criteria)

Sct i = 1 (initialize iteration counter)

Step 2. The search direction is obtained as a solution to

[A;]S， = -Vf(X，) x川=~・+Cl，S，; dJ( =α，S， αi is detennined by Min旬並zingf(X川)

Step 3. If Vf(X川)TVf(X川)三E3・converged

If Ir(Xi+l)ーf(X，)1~ EI・sωp (function not chunging)

If !1XT dJ( ~ E2; stop (design not changing)

If i + 1 = N， stop (iteration limit)

Else

Y = Vf(Xi+l) -Vf(X，)

yyT [B) =ーτ一一

y'dJ(

Vf(X，) Vf(X，? [ C ]=中

Vf(X;)'Sj

[Aj+d = [A;] + [B) + [c]

if-ー i+1

GoTo Step 2

Applic沼tlon0' BFGS Method: The method is applicd to Example 6.1. Only cssential computations are shown.

cpI XI=lsl; 川 1)= 5.31お;凶1=[~ ~l Vf(X1)七二lWSl=lzl; 叫 =0.19耽 X2=ltzzl;Vf叶協'~]

step3Y=12Jri;m=ltml

6.4 NUMERICAL TECHNIQUEs-5ECOND ORDER 251

山一地

制州一附

RU

i立 Eぷふcl=|2m-3例21-3.0429 6.5095

削 S24221; 町=1お44; X3=[H 胸 =3.0; 帆 )=[g]

Converged

The translation of the BFGS algorilhm into codc is left as an exercise for the student. It is recommcnded that Ule calculation of the search direction in Stcp 2 be accomplished as a solution to a set of linear equations (as listed) raUler than inverting the matrix A. One other thing to note for Example 6.1 is that if the polynomial

approximation was used for the one-dimensionul process， the calculation of α-would have becn exuct (why?).

6.4 NUMERICAL TECHNIQUES-SECOND ORDER

S巴cond-ordermethods for unconstrained optimization are not uscd b巴caus巴巴valualionof the Hessian matrix during each iteration the Hcssian matrix is considered to be computationally expensive. A second-order method with白eproperty of quadratic convergence is very impressive. An n-variable problem can convergc in one iteration. As mentioned before， for real design problems， where decisions are required to be made on Ule discrete nature of some of出evariables，出巴 existenceof the first derivative， not to mention second derivatives， is questionable. Moreovcr， Ule

quasi-Newton methods of由巳 lasttwo sections are able to effcctively function as

second-order methods as出eyapproach批 solution-andthey do not need the estimation of second derivatives. One second-order method is prl巴sentedhere for the

sake of completen巴ss.Indeed， there is only one basic second-order technique for unconstrained

optimization. It is based on the extension of the Newton-Raphson technique to multivariable problcm. Many diffcrent cxtensions are available， but in this text， a

253 6.5 ADDITIONAL EXAMPLES

(iteration Iimit) stop If i + 1 =N，

Else

i←i + 1

GoToSt巴p2

Application of Modified Newton Method: Algorithm (A6.9) is used to solve Example 6.1. Only essential computations are included:

f(Xt) =5.3125

-EEE

・E

・-E」

∞∞ 0

5

弓M

今

&

nU

今ム

「トート一一

QU

s， =1 ~.;~~~ 1: u， = 1.0: X? =1 ~ 1 一¥7'~~;;;; ¥; α1=1.0; Xヮー¥;¥ 1 1.5000 l' Uot -J .v， "~2 -1 2 1

Asexpected only a singleiteration wasnecessary toobtain thesolution. Translation

ofthealgorithm intoMATLAB code is once again left as an exercise.

converged f(X2) = 3.0;

Three additional examples are presented in出issection. With many of the numerical

techniques already programmed， obtaining the solution is a matter of judicious

application of the numericaI procedures. This section also presents some creative use of the methods and explores nonquadratic problems.百lefrrsl example is出e

Rosenbrock problem [9].百lIsexample was among those created to cha11eng巴白enumerical techniques for unconstrained minimization. It is also sometimes referr芭dto

as the bananafunction. Ifyou have a copy ofth巴Optimi<.ationToolbox from MATLAB， you wi1l see山isas part of出巴 toolboxdemo. You wiJ1 also note山atthe toolbox contains many of出巴 techniquesdeveloped in this section. The second example is a solution to a nonlinear two-point boundary va¥ue problem that is due 10由eNavier-Stokes

叫uationsdescribing flow due to a spinning disk.百1巴lastis an unusua¥ data fi凶ngexample using Bezier curves. Once again the Optirnization Toolbox presents its version of the data fitting example when running optdemo at the MA TLAB prompt

ADDITIONAL EXAMPLES 6.5

(6.15)

Example 6.2-Rosenbrock Problem

f(xl・X2)= 100(x2 -xT)2 + (1 -XI)2

-2::;;xl ::;;5;

Step1431;

抑協izl

-2::;;X2::;;5

6.5.1

The Rosenbrock problem is

Minimize

Subject to:

よ軍部慰霊33準3333却損

41131411ffits?hititi--現1iiftJ

NUMERICAL TECHNIQUES FO円UNCONSTRAINEOOPTIMlZATION

direct extension recast in出egeneral algorithmic s釘uct町巴 (A6.1)is presented. Once

again the general problem and specific example are as fol1ows.

252

(6.1) f(X); [X]n

xl三Xi三xjt

Minimize

(6.2) i= 1，2，... ，11 Subject to:

(6.3)

(6.4)

f(X) = f(XI' X2) = 3 + (XI -1.5x2)2 + (X2 -2)2

O::;;XI:豆5;0::;;x2::;;5

Minirnize

Subject to:

Modifled Newton's Method: The Newton-Raphson method， used for a single variable， solves the problem中(X)= O. This equation can問present白eFOC for an

unconstrained problem in one variable (Chapter 5). The iterative change in由evariable is computed through

(6.13) dx=ー単王L=ーと主主ゆ'(X) f"(x)

where f is th巴 single-variableobjeclive function. The multivariable extension lo computing a similar change in出evariable veclor is

(6.14)

The originaI Newton-Raphson is not known for its robustness or stability and

Equation (6.14) shares the same disadvantage. To control the design variable changes and to bring it under由eschem巴ofa1gorithm (A6.l) the left-hand side is defined邸

the search direction v巴ctorS followed by a standard one-dimensionaI stepsize

computation. This is terrned the modified Newton method. The complete algorithm is as follows.

ÅX= ー[問ー'~f(X)

Algori的m:Modlfled Newton Me的od{A6.9)Step 1. Choose XJ， N

E" E2' E3: (tolerance for stopping criteria)

Set i = 1 (initiali7..e iteration count巴r)

Step 2. The search direction is obtained as a solution to

[H(Xj)]Si = -Vf(X1); [町 is出eHessian

Xi叶 =Xi+α，/Si; AX=α.，SI α"i is deterrnined by MinimizingJ(Xi+l)

Step 3. If Vf(X川 )TVf(Xi+I)三E3' converged

If Ir(X仲 1)ーf(Xi)1::;; EI' stop (function not changing)

IfAXT sX::;;E2・SlOp (design not changing)

254 NUMERICAL TECHNIQUES FOR UNCONSTRAINED OPTIMIZATION

The side constraints are used for drawing contours. The solution to this problem is

x; = 1.0; x; = 1.0; I = 0.0

Thc problem is notorious for 犯quiringa large number of ilerations for converg叩 ces.

Changes in design are small as the solution is being approached. Here the Random

Walk and Conjugate Gradienl methods are used to examine the solution. Both these

methods are changed 10 draw appropriale conlour levels. To avoid c1utter， disable

printing lexl information on批 linesrepresenting design changes. Also， sinc沼山ese

methods are run for 200 iterations， the fmal printing 10 the Command window is avoided. The trac巴ofthe design variables on the contour plot is provided as the outpul

from lhe program， along wlth Ih巴linalvalues. As the figurc is being drawn， it is c1ear that the progress to the solution is slow.

Solutlon Using Random Walk Method: Figure 6.7 presenls the resull of

executing RandomWalk.m for thls example. Th巴programinvocation and output from

曲eCommand window is

N M

-1 。 2 3 X 唱

Flgure 6.7 Random walk method: Example 6.2.

4 5


>> RandomWalk('Example6_2'， [3 2] ，200，l.Oe-08，0，l，20) The problem: Example6_2

ans 2.0000e+002 1.0000e+000 1.0000e+000 1.2327e-007

Since th巴reare no stopping criteria bui1t into the method (was suggested as an exercise)， the method ran for 200 ilerations (first value).明tefinal value for the design V削 orand山eobjective function are very impressive:

random walk : x; = 1.0; x; = 1.0; I = 1.23e -07

Solution Uslng Co呼ugateGradlent Method: The invocation and output from the Command window is

》∞njugateGradient('Example6_2'，口 2)，200，1. Oe四 08，0，1，20)The problem: Example6_2

No. of iterations: 200

ans 9.6B07e-001 9.3700e-001 1.0222e-003

The solution is

conjugate gradient : x; = 0.96807; x; = 0.937; r = 1.0222e -03

This appears no match for the Random Walk method for出isinvestigation. Figure 6.8

tracks由edesign changes on the contour plot.

6.5.2 Example 6.3ーThree-DlmensionalFlow near a Rotating Disk

This example represenls another of Ihe exact solutions to出cNavier-Stokes equations from fluid mechanics. The nonlinear two-point boundary value problem describes a viscous flow around a flat disk出atrotates about四 axisperpendicul釘 toits plane wi出a constant angular velocity. After suitable 児 defmition[10] the mathematical

description of the problem is the following set of coupled nonlinear equations with

boundary conditions at two poinls:

2F+H'=0 (6.16a)

F2 + F' H -G2 -F" = 0 (6.16b)

2FG+ HG' -G" = 0 (6.16c)

256 NUMEAICAl TECHNIQUES FOR UNCONSTRAINED OPTIMIZATION

... M

。x t

2 3

Flgure 6.8 Conjugate gradlent method: Example 6.2.

4 5

F(O) = 0; G(O) = 1; H(O) = 0 (6.l7a)

F(∞) =0; G(回 )=0 (6.17b)

Equations (6.16) are convertedωstate spa回 fonn(s白 Chapter5， Example 5.3). The design variables arE the missing boundary condi6orts on F'(O)，G'(O)and the final

value of the independent variable (in lieu of∞). The objective function is出esquar吋crror in the integrated vaJue al the final condition.

The Optlmlzation Problem

Minimize f(xl' x2・x3): YI(X3)2 + Y3(x3)2 (6.18)

where thc slalc variab1es are oblained from the solution of thc following initial value problem:


YI Y2 YI(O) O

Y2 ， Yi+Y尚一え Y2(O) ~II ， Y3(0)

(6.19) Y3 ー Y4

Y4 2YIY3 + Y，v'5 Y4(O) x2

Y; -2YI Y5(O) 。

Thc statc equations (6.19)釘.cintcgraled using MATLAB function ode45 .

Solutlon to the Problem: The three-variable problem is solved using the DFP method. Example6_3.m r芭tumsthe objective function. It calls the function

Ex6_3_state.m which rcturns the system巴quationsin state fonn. The call to the

DFP method with白einitial design vector of [1 -1 8J with 20 iterations of the DFP

method， and a tolerance for the golden section of 0.0001 is shown below. AJso in the

call is the input to lhe upper bound calculation.

Usage: DFP (' Example6_3' ， [1 ・ 18]，20，0.0001，0，1，20) (6.20)

The final value for the design after 20 iterations was

x;(ゆ)= 0.5101; .t;(t.tか)= -0.6157; xj(併ゆ=10.7627; j = 5万 08e心06(6.21)

111e tolerance for the golden section was only 0.0001. h is possiblc lhc objectivc

cannot decrcasc unlil a small巴rtolerance is specified.

The published values for the design variables ar巴

x; = 0.5101;弓(物)=-0.6159 (6.22)

which is remarkably close to Ihe values obtained by the DFP method. The vaJue of

xj is ILslLally not specified bUI is about 7. Since it is a substitute for infinity， the Jarger number should not be a probJem.

Example 6.3 is not trivial problem. The integration is highly sensitive to the

initial values. A reasonable starting point is essential to prevent the integration

from generating NaN's (nol a number). The second design variable musl have a

ncgative sign. The exampl巴 illustratesthe application of standard optimization

technique to soJve a nonlinear differential system. A similar appJication can be

made to problems in syslem dynamics and optima1 control. It is essential to

understand白紙 thealgorithms outlined here are numerical too1s出attranscend any

particuJar discipline. There are numericaJ techniques山ataddress two-point non1inear boundary value

problems more efficiently. However， computing resources， especially on a PC， are not a concern. Therefore， t11e proccdure adopted in由iscxample is an acceptab1e approach

258 NUMERICAL TECHNIQUES FOR UNCONSTRAINEO OPTIMIZATION

for this class of problems. Even the Random Walk method is impressive in出is

mstance.

6.5.3 Example 6.4-Fittlng Bezier Parametrlc Curves

The example introduces a nontraditi叩 aland an unusual curve fiuillg procedure.

Given a set of data points a Bezier curve 01 user-chosen order is fil 10 the data. Bezier curves are parametric curves自制御官 aspecial case of uniform B-splines. Bezier

parametrization is based on the Bemstein basis functions. Any point P on a

Iwo-dimensional Bezier curve (any parametric curve) is aClually obtainedぉ P(x(v)，y(v))， where 0 S v S 1.百leactual relations are

P尺(か例v吟)=[ドxy升)=エB却Jんn肌即.1川(いv;=0

JII.;(v)や)ゾ(1-ν)".1

(6.23)

whereB;紅E白evertices of出epolygon that determin巴sthe curve. B; represent a pair

of values in two-dimensional space. The order of the curve is lI-th巴highestpower in lhe basis functions. The aclual computations of the poinlS on Ihe CUlveω'e ellsier

using matrix algebm and are well explain巴dand documented in Reference 11. Figure

6.9 represenL~ a cubic B巴ziercurve. The following are some 01' lhe most useful

prop凶 iesof the curve as obs巴rvedin the figure:

• The curve is completely defined by the polygon obtaincd by joining the vertices

in order.

・τ'hedegree of由epolynomi必definingthe curve is one less出anthe number of

verti白 sof the polygon.

. The firsl and last points of the curve are coincidcnt with the first and last vertex.

Thc remaining vertex points do not lypically Iie on thc curvc.

. The slopcs al the ends of the curvc have the same direction as the sides of the

polygon

. The curve is contained within the convex hull of山epolygon.

In this ex創nplethe convenient matrix representations for thc curve arc used. They are not discussed here， however. but are availablc in the codc. For由isexample. the dcsign

variables are the inside polygon vertices. For n = 5， thcre arc 6 ve凶cω.The first and

last vcrtex are known from the data points. This leaves 4 vertices or 8 d巴signvariables

forn = 5.

Example6_ 4.m:百lism-fiIc is a stand-alone program to run thc cxamplc in this

scction. Typing Example 6_4 al the prompt should start thc program. Actually the

0.8

0.7

0.6

圃

805

p、

0.4

0.3

0.2

0.1

。

6.5 AOOITIONAL EXAMPLES

'1

ー"82 /， a ノ'，、/，，、 /i

f ¥/， .、~， ./'町. 、 ~今占

-、~1 、 J 〆，.，，、---ー， ~同.一，， ~、，

1 ~.、， ~〆、，

1 / "、，1 . /、 l'

1 . /、.1 / 、 J/ 、，， / 了

.' / C!I 1/ ，ノー四一-，/ 占 E3' "ふ

B唱 φ

ーームー0.2 0.4

Flgure 6.9

x data

Acublc B自zl日rcurve.

1.6

259

2

example is generic as most decisions a陀 basedon白etext file that contains the data

to be fiued. 11 calls on m叩 ym-files. The Bezier caIculations are fairly modular. This

program is not凶vial.A more important隠 asonfor its inclusion is白at，if understood，

it signals a reasonable maturity with MATLAB programming. More importanlly， it exemplifies出eway to exploil the immense power of built-in MATLAB suppo目.The

author encourages the user to exert somc effort tracing the progress of the calculations

血rough白ecode. A 101 of program control is based on出euser' s choice conceming

the order of the Bezier curve. Also the initial values for the design variables are

generated automaticalか"based on the data 10 be fitted. The function calling sequence

used in出isexample can be bettcr understood by the following structure:

Example6_5_3.m CaUs uigetfilc to pick up山e(* .txt)日Ic山atcontains two-column data白atwill be

fit (Ex6_5_3.txt)

τ'hrows up an inputdialog box to obtain the order of fit from the user (0 = 5) Calls coeff.m with the order information

Calls comblnatlon.m -calculation 01' combination for Bcrnstein basis

Calls Factorial.m -caIculates factorial of an integer

260 NUMERICAL TECHNIQUES FOR UNCONSTRAINED OPTIMIZATION

Calls Curve_fit.m which handles the optimi祖 tionCalIs DFP CalIs Bez一Sq_Err.m一calculationof O~対je郎:ct“iv刊e function Calls gri悶'sdf伽un.m-calculation of deriva低叫tiv巴s Calls Gold_Section一nVa創rι川'.m官首官 - 1 D stepsize

Calls UpperBound_nVar.m -bracket the minimum

Since there are eight design variables， a11 of the information except for iteration number. convergence criteria， and白巴 obj巴ctivefunction is suppressed from the Command window to prevent scrolling and loss of comprehension.

百lesignificant elements in出iscodear芭 asfollows:

• Picks up白edata file througb a file selection box.

• Uses an input dia10g box to query the user on the order of the c町 ve.

• Theord巴rdetermines the size of the coefficient matrix A and it is automatica11y generat凶.

• The number of design variables is automatically determined. Once again it is basedon曲巴orderof the curve.

・ The initial values for the design are automatically generated based on the da胸to be fit.

・TheDFP method is used to find the optimum.

• The objective function is computed in the generic fiIe BeziecSq_Err.m. The data specific to出e巴xampleare passed through出eMATLAB global statement.

・The original data set and白巴 Beziercurve fitting the data ar，巴 displayedat the end.

・Theobj配 tive function and th巴convergencedata for each iteration are printed to the Command window.

ObJectlve Functlon: Th巴objectivefunction is the 1開 stsquared error over all of the data points between出eactual data and the fitted Bezier curve. It is expr'巴ssedas

Minimize: IIDa佃

六X):エIy，帥 -YBT1<=1'" .J

(6.24)

The following information is copied from the Command window (the file is Ex6_5_3.txt and出eorder is 5):

持 Bxample6_4iteration number: 1 gradient length squared: objective function value: iteration number: 2 gradient length squared:

5.0001 0.5302

5.6890


objective funcヒionvalue: 0.2497 iteration number: 3 gradient length squared: 0.0546 objective function value: 0.1830 iteration number: 4 gradient length squared: 0.1322 objective function value: 0.1605 iteration number: 5 gradient length squared: 0.0573 objective function value: 0.1467 iteration number: 6 gradient length squared: 0.0374 objective function value: 0.1437 iteration number: 7 gradient length squared: 0.0071 objective function value: 0.1433 iteration number: 日

gradient length squared: 0.2793 objective function value: 0.1415 iteration number: 9 gradient length squared: 0.4337 objective funcヒionvalue: 0.1401

UI From the data above it appears that the e町oris around 0.14 and is difficult to rαiuce further. It also appears the reduction in the objective is accompanied by the increase in the gradient. after a certain number of iterations.lt is moving away from satisfying the FOC. The best value for the FOC is about 0.007 and starts to worsen with further iteratIons. It is quite possible白紙numericalinaccuracies may be a problem as there are巴xtensivecomputations involved in each iteration. It is also likely白at白enumerica1 gradient computation may need to be refined. Currently出egr叫 ients釘ecomputed using forward finite difference with a step ofO.OO1. These are parameters tltat n田 dto b巴巴xploredin optimization as there is no universal set of values for all classes of problems.

Figure 6.10 illustrates出eorigina1 data叩 dthe Bezier curve a白er出elast iteration. The polygons are not shown as our effort is to gen巴ratethe c町ve.τhefit is definitely acceptable and there is probably little白eBezier curve can do about a couple of points that are not very smoothly located. It is probably these points that do not a110w the objective functIon to decrease further without increasing th巴gradient.This concludes another innovative use of unconstrained optimization.百leauthor has used these curves to optimally design airfoils for low-speed flight.

'" '11'

叩訓Id--R崎浦W岨43

』ドヨ丸山

tmitt

H

E

l

i

-

-

Note on FOC for Unconstrained Minimization:明白 examplehas alerted us to the question of FOC Vf = O. If analytical computation of the derivative is possible for a problem. then FOC has the likeliltood of being met. ln design problems like Example 6.4， where the elements of the gradient vector have the form

51且

ゐ

‘ 、h 、h 。、、、

弘、α、o、o

唱。‘・ 4

可)o.e..._ 0 σ-e ...旬。

三三土92

263

5. Fletcher， R.， and Reevcs， R. M.， Function Minimization by Conjugale Gradients. The ComputerJoumal. Vol. 7， pp. 149-180， 1964.

6. Davidon， W. C.， Variable Metric Methods for Minimizalion， U.S. Alomic Energy Commission Research and Development Reporl No. ANL -5990， Argonne National Laboratory， 1959.

7. Huang， H. Y.， Unified ApproachωQua命凶caJlyConvergent A1gorithms for Function Min回国.tion，Joumal of Op肋uwtIon刀'aoryand Applicatio/ls， Vol. 5， pp. 405-423， 1970.

8. Vanderp1aats， G. N.， Nl/lnerical Optimizatio/l Techniques for Engineering Design， McGraw-Hil1， New York， 1984.

9. Hock， W.， and Schittkowski， K.， Test Examples for Non Linear Programmil!g Codes， Lecture Notes In Economic and Mathematical Systems， 187， Springer-Verlag， Berlin， 1980.

10. Schlichting， H.， BoundaηI Layer Theory， McGraw-HilI， New York， 1979.

11. Rogers， G. F.， and Adams， J. A.， Mathematical Elementsfor Computer Graphics， 2nd ed.， McGraw-HilI， New York， 1990.

PROBLEMS NUMERICAL TECHNIQUES FOR UNCONSTRAINED OPTIMIZATION

Ed-t

au-

du

一f

o-:

蜘附一

川印一

aEF--

n"au-

-邸岨一;

OB-

hv--

O一一

4圃

一

ー

262

e

aa守句

a

q

，‘

{SEEE幅窃

帽

巴

)S咽司・』同

PROBLEMS

Apply the Random Walk method to Example 5.2.

Solve using the Random Walk method:

Minimize !(X.，x2) = x1 -2み2+ x1+~-2x. +4

Apply the Pattem Search method to Exampl巴5.2.

Solve using the Pattem Search m巴白od:

Minimize !(.x.，x2) = x1-2x1X2 + xi +.q -2x. ... 4

Modi均， the PatもemSearch so that program control p紅白netersfor contour

plotting， golden section， and upper bound calculation can be set by由巳国er.

Include曲巴 promptand allow for default values in case the user decides not to

take advantage of it.

Translate the Powel1 method into working MATLAB code. Verify the solution to

Examples 6.1 and 5.2. Start fTom several points and verify that the number of

cycles to converge is出esame.

Verify白紙曲esearch directions are conjugate with児 spectto the Hessian matrix

for Exarnples 6.1加 d5.2.

Solve using Powell's m巴山od叩 d白atverify any two search directions are

co吋ugatewi白 respectto the Hessian matrix at the solution:

Minimize !(.x.，x2) = x1-2x1X2 + xi +.q -2x. + 4

Solve Example 5.2 using the Steepest Descent， Conjugate Gradient， DFP， and

BFGS me曲ods.

6.10 Solv巴 usingthe Steepest Descent， Conjugate Gradient， DFP， and BFGS methods自己 problem

附凶凶ze f(x. OX2) = x1-2x1X2 + xi +.q -2x. + 4

6.6

6.8

6.9

6.7

6.3

6.4

6.2

6.5

6.1

2.5

and which are best computed numerically， some specifications are needed regarding derivati ve computations if they are to be used for enforcing the necessary conditions.

REFERENCES

1.5 x・data(read from flle)

Orlglnal data and Bezler curve.

怖いi)

1. Fletcher. R叫 PracticalMetllOdsfor Optimiwtion， Vo1.1， Wiley， New York， 1980.

2. Brent， R. P.， Algorithms for Millimization without Derivatives，加問tice-Hall，Eng1ewood Cliffs， NJ， 1973.

3. PowelJ. M. J. D.， NOlllinear OptimiWlion， Academic Press， New York. 1981.

4. Fox， R. L.. Optimizatioll Methodsfor El!gineering Design. Addison-Wesley. Reading，

MA.1971.

Flgure6.10

0.5

11-

。。

264 NUMERICAL TECHNIOUES FOR UNCONSTRAINED OPTIMIZATION

6.11 Verify the values in S巴ction6.3.3 using a calculator. Verify if the matrix A at

出esolution is山einverse of the Hessian.

6.12 Verify if the matrix A at the solution is the inverse of the Hessian for DFP

me出odfor

Minimize f(XI，x2) =x1-2xtx2 +xJ +x~ -2x1 + 4

6.13 Develop山eBFGS method into MATLAB code and verify the caJculations in

Section 6.3.4.

6.14 Develop the Modified Newton mcthod into MATLAB code and apply it to

Example 5.2.

6.15 How will you incorporate白eside constraints into the cod巴foraJl of the various

methods in this section? Implemcnt them in thc numerical procedures.

6.16 Solve Examples 6.2， 6.3. and 6.4 by one other m巴thodof the section.

6.17 Identify and solve a system dynamics problem using any method ofthis section.

6.18 Identify and solvc your own curve fit problcm.

7

NUMERICAL TECHNIQUES FOR CONSTRAINED

OPTIMIZATION

百lischapter explores aJgorithms/methods that handle出egcneral optimization problem. Both equ叫ityand inequaJity constraints are included. For an engineering probl巴mthis will involve bringing several nonlinear relations into the design space.

This is certain to i即応asethe degree of difficulty in obtaining出csolution. For the designer山ereis an additional burden of being more attentive to the design changes

and the corresponding numbers to coax the solution if the mathematical definition of the problem is parlicularly severe. In aJl of lhese problems lhere are two outcomes that

出巴 algorithmsseek to accomp¥ish.百lefirst is to ensure曲atthe design is feasible

(satis自由aJlconstraints) and the second伽 tit is optimal (satisfies the Kuhn吊 lcker

conditions). While the focus is on determining曲esolution， in times of difficulty， it is essential 10 remember Ihatfeasibili砂ismore important出atoptimality. Also unstated in出巴 restof白echapter is山at出eoptimal solution must be feasible. Two distinct

approaches wi11 be used to handle the constrained optimization problem. The first approach is termed出eindirect approach and solves山巴 problemby transforming it

into an unconstrained problem. The second approach is 10 handle出econs凶 ints

withoul回 nsformation-thedirect approach. The indirect approach is an expression of incremental development of the subject

to take advantage of出ecurrent methods. For ex創nple，it leverages the DFP method to handle constrained optimal problems. Two indirect methods are presented. th巴Exterior Penalty Function (EPF) method， and出巴 AugmentedLagrange Multiplier (ALM) method. The direct approach handles the constraints and the objecti ve together wi山outany凶 nsformation.Four methods蹴 p陀 sented.百lemethods in出isbookare

SequentiaJ Linear Programming (SLP)， Sequential Quadratic Programming (SQP)， Generalized Reduced Gradient Method (GRG)， and SequentiaJ Gradient Restoration Algorithm (SGRA).

265

266 NUMERICAL TECHNIQUES FOR CONSTRAINED OPTIMIZATION

7.1 PROBLEM DEFINπION

The standard format of the nonlinear programming problem (NLP) is reproduced he陀

for convenience:

Minimize f(x" X2， • • • • Xn) (7.1)

Subj配 tto: hk(x" X2' . . . • Xn) = O. k = 1，2， • • • .1 (7.2)

gj(X" X2.・・.. Xn) ~ 0， j = 1. 2， • • • . m

XISXj~xll i=I.2.....1t

In vector notation

Minimize f(X). [Xl"

Subject to: [b(X)]f = 0

[g(X)]"， S 0

X1nwSX:;;X"1'

(7.3)

(7.4)

(7.5)

(7.6)

(7.7)

(7.8)

For this chapter the following indices are rcserved: It. number of variablω; 1. number of equality constrainls; m. number of inequality constraints. Many of Ihe algorithms and methods present a reasonable level of difficulty. A two-variable prob1em (for graphical i1lustration). with a single equality and inequality constraint is used to illustrate the various algori出ms.Al1 of the functions are nonlinear a1though the constraint functions have simple geometric shapes.

7.1.1 Problem FormuJatlon-Example 7.1

Example 7.1 is a simple mathematical formulation. It does not陀 presentany engineering prob1em. It is cons凶 ctedto have a solution of xj = 1叩 dxi = 1 for恥

∞nstrained prob1em and xj = 0.8520 and xi = 0.8520 for the uncons回 inedprob1em. The two constraint functions have simple non1inearity. The叫ualityconstraint is a ci四lewhile the inequality constraint is an e¥lipse.

Minimize f(x" X2): x1-2x ~X2 + xr + XI x~ -2x1 + 4 (7.9)

Subjecllo: h(X"X2): xr+xi-2=0 (7.¥Oa)

g(x" X2): 0.25 xT + 0.75 x~ -1 S 0 (7.lOb)

O:5xl :55; 0:5x2:55 (7.¥Oc)

F 口信


9<0

3 4

F1gure 7.1 Graphic剖solu1ionfor Example 7.1.

Figure 7.1 displays the graphical solution to出巴 problem.It is clear from the figure that the solution is at xT = 1. xi = 1 (why?). The value of the function is 3. A simple a即 roachto the solution of the problem is 10 u印刷uation(7.lOa) 10 eliminate one of曲evariables針。m山eproblem由erebyreducing it to a single・variableconstrained problem. 百lisis not done he陀 butit is a good exercise. FIg7 _l.m 1 will create the figure.

7.1.2 Necessary Condltions

引le necessary condition-or the Kuhn-Tucker (KT) conditions for 由eproblem-was devcloped in Chapter 4. It is based on the minimization of出ELagrangian function (円.The condilions are

Minimize

F(xl .. 'Xn' AI ... Af' sI ... sm) = f(xp…xn) +λIhl+…+λ内+

Pぱ1+・・・+s"，g"， (7.11)

1Ft!田10be downloaded from白ewebsi旬 areindicaled by boldface sans serif Iype.

A4

「ぞL/

269

(7.l7c)

Equation s巴t(7.17) is solved symbolically in MATLAB. Thcre are two solutions tha! satisfy the sidc cons回 ints(7.l0c)

PROBLEM DEFIN汀'ON7.1

xi +xi-2=0

NUMERICAL TECHNIOUES FO円CONSTRA'NEDOPTlMIZATION

subj配 tto山econs汀aints.百le陀 aren + 1 + m unknowns in the Lagrangian. The same

number of equations is requII吋 tosolve出巴problem.These are provided by the FOC

or Kuhn-Tucker conditions. Here'l equations are obtained as

268

j= 1.2..... n

(7.12)

dF dr • dh. . • dh， • 0 d81. . 11 d8m

一=~J +λ」+ー・+L-41」+ー・+P，JS7=0;axi axaIaxttaxi t axi dxt x; = 1.4142;

x; = 1.0676;λ・=-0.1217

λ・=0X; =0;

x; = 0.9275;

80th of these violate出einequalily constraint. In Figure 7.1. the first solution definitely do凶 butlhe second is a liltle difficult 10 see. Zooming should help. Case a docs not provide a solution.

1 equations are obtained directly t1lfOugh由eequality constrainls

(7.13) k= 1.2.....1 hix，・X2.....xn)=O;

百lisestablishes four equations as g=O Case b:

8j<0 then

m cquations are applicd through the 2m cases

sj8j=O→if鳥=0(7.18a) 4xt -4x内 +2x， -2+x~+2 λ x ， +0.5 sX1 =0

(7.18b) -2xf+2x，x2+2入X2+1.5 ß~=O (7.14) 島>0lhen if 8j=0

(7.18c) xi +x~ -2 =0

The KT conditions apply only if the points are regular (see Example 4.3).

(7.18d)

Equations (7.18)脱 solv叫 symbolicallyand lhe only solution satisfying Ihe side constr剖nts，s

0.25 x~+O万x~ -1 =0

s"=O

which was expected (for the design variablcs not lhe multiplicrs). The valuc of s" = 0 is not expecled as lhis is an active constraint. The KT conditions can be considered 10 be weakly salisfied as s is not strict1y positive. As in Chapter 6. wherever possible. 出eKT conditions are used to check for convergence.

i.." =-0.5000; 弓=1.0; xi = 1.0; a

;

t

i

-

-

Appllcation 0' the KT Conditions:百 eLagrangian

F(x" x2• λß): x~ -2xT x2 +xT+x， x~ -2x， + 4+

λ(ヰ +~-2)+ 戸(0.25xi + 0.75 ~ -1) (7.15)

(7.16a)

(7.16b)

Three equations are set up through

dF E;=44-4xtX2+2XI+xi-2+2hi+05PXl=0

dF ァ=ー2xT+2x，x2 +2 i..x・2+ 1.5 sX2 = 0 C1X2

Elements of a Numerical Technlque

There is a distincI di仔erencein the numerical techniques with incorporation of the.

constrainlS. even if lhe idea of using a search direction veclor is still useful. The major differences in the a1gorithms of the previous chapter was山巴waythe search directions

were established. To understand the necessary changes. the general approach for unconstrained optimization as seen in the previous chapter was:

7.1.3 (7.16c) h =xi +x~ -2 =。

With this information Equations (7.16) become

The fourlh equation is deve10ped through

s=O Case a:

General Algorlthm伺6.1)

Step 1. Choose Xo

(7.17a)

(7.17b)

4x1 -4 X， x2 + 2x1 十 x~-2+2 λ x， =O

-2xT + 2x， x2 + 2λx2 =0

270 NUMERICAL TECHNIOUES FOR CONSTRAINED OPTIMIZATION


Detennine search direction vector Sj

(It would be nice if the objective decreased along this direction)

Step 3. Calculate AXj =α，Sj Note: AXj is now a function of the scalar (Xj as Sj is known from Step 2

(Xj is called白estep size as it establishes白elength ofAXj

a; is deterrnined by Minimizingf(X;+I)' where

Xj+1 = X， + AXj

As the above steps釘 ereviewed， in Step 2 a dilemma n民 dsto be resolved: Should the search direction Sj decrease出eobjective function 0' should it attempt to satisfy the constraints? (It is assumed出atit cannot do both though very often it will.) Since

feasibility is more important出anoptimality， it is回 sierto identify the latter as a

significant influence in deterrnining the direction. This is more difficuIt than it sounds as each constraint can have its own favorite direction at the current point. Such

occasions call for a trade-off and to see if it is possible to negotiate a decrease in the objective function as well. It is clear that Step 2 will involve investment of effort and

some sophistication. Step 3 is not far behind in the児 qui回 mentforsp田 ialhandling. Once the direction

is established (Step 2)， what kind of stepsize will be acceptable1 Several results are possiblc. First， thc objcctive function can b巴decreasedalong the direction. Second， current active constraints (including由巳 equalityconstraints) can become inactive， violated， or can still p陀 serve白eactivc state. Third， current inactive constraints can undergo a similar change 01" state. Fourth， probably most important， current violated cons釘aintscan become active or inactive. It is difficult to encapsulate this discussion in a generic algorithm. Somc algorithms combine Steps 2 and 3 into a single step. Some divide the iteration into a feasibility componenl and an optimality component. In essence the generic a1gorithm for白ischapter， which is not particularly凶 eful，is:

Generlc Algorlthm (A7.1) Step 1. Ch∞seXo


Detennine search direction vector Sj

Step 3. Calculate AX; =αjSj

Xj+1 = Xj + AXj

7.2 INDIRECT METHODS FOR CONSTRAINED OPTIMIZAT10N

These mc出odswere developed to take advantage of codes白atsolve unconstrained

optimization problems. They are also referred to as Sequential Unconstrained

Minimization Techniques (SUMT) (1). The idea behind山eapproach is to repeatedly

call the unconst，ai1/ed optimizatio1/ algorithm using the solution of the previous

7.2 1NDIRECT METHODS FOR CONSTRAINED OPTIMIZATlON 271

rt古若干抱沼

44j532

iteration.百leuncons住ainedalgorithm itself executes many iterations. This would

r叫ui陀 arobust unconstrained minimizer to handle a large class of probems. The BFGS method is robust and impressive over a large class of problems.

A preprocessing lask involves tr叩 formingthe constrained problem into an

unconstrained problem. This is largely accomplished by augmenting the objective

function with additional functions白紙 reflect白eviolation of the ∞nstraints very significantly. These釦nctionsare referred to as由epenal，ηf附 ctions.百le印 Wお

significant activity in山is紅白 atone time which led to entire farnilies of different types of penally function methods [2). The firsl of these was the Ex1erior Penalty Function (EPF) method. The first version of ANSYS [3]白紙 incorpora1ed

optimization in its finite elemenl program relied on the EPF. The EPF (presented in 白enext section) had several shortcomings. To address those出eInterior Penalty Function (IPF) methods were developed leading 10 the Variable Penalty Function

(VPF) methods. In白istext.， only the EPF is addressed largely due to academic

interest. In view of白eexcellent performance of出巴 directmethods， these methods

will probably not be uscd today for continuous problems. They arc once again important in global optimizatiorl techniques for constrained problems. The second

me白odp民sentedin this section， lhe Augmented Lagrange Method (ALM)， is lhe besl of the SUMT. lts exceedingly simple implementation， its quality of solution， and its ability to generate infonnation on the Lagrange multip1iers allow it to seriously

challenge由巳directtechniques.

7.2.1 Exterlor Penalty Functlon (EP町Method

The transformation of thc optimization problem (7.1)ー(7.4)10 an unconstrained problem is made possible through a penalty function fonnulation. The transfonned

unconstrained problem is:

Minimize F(X， '11' 'g) = f(X) + p(X， '11， '，) (7.19)

(7.4) X::S;X/:S;ヰ i= 1， 2， • • • ， 1/

where P(X， 'h， 'g) is the penalty function. '11 and 'g are penalty constants (also called mult与Jlie，s.)

The penalty function is expressedぉ

p(X， 'h' '，，) = 'h 1 L. "k(X)21 + '81ヱ(m阻 {O・め(X)})21 (7.20) 1"'1 lt1

In Equation (7.20)， if白eequality cons回 intsare not zero， their value gets squared and multiplied by the penalty multiplier and then gets added to白eobjective function. If th巴inequalityconstraint is in violation， it too gets squared and added 10出巴 objective

function after being amplificd by出巴 penaltymultipliers. In a sense， if the constraints


are no1 satisfied， then由eyarepen必ized，hence the function's n創ne.It can be shown白紙

thc四 lsfonncdun∞ns凶 n吋 problemsolves出eoriginal∞m凶 nedproblem as the multipliers r"， rg approach∞. In order for P to remain finile at山ωevalues of山emultipliers (1即 dcdfor a valid solution)，出econstraints must be satistied. In computer implementation血islimit is陀placedby a large value ins飽adof ∞. Another facet of ∞mputer implementation of this method is白紙alarge value of the multipliers at the白百tileration is bound to create numerical diffi印 lties.百lesem叫tiplier百arestarted with smal1 values田 du凶a凶 gωme凶回lly明白回chiteration.百 eunconslraIn吋飽chnique，for ex胴体 DFP，wi1¥ solve Equation (7.19) for a known value of恥 multiplicrs.The solution剛山ncdfrom the DFP can be consider凶 asa function of thc multiplier and can be thought of as

X. = X.(r"・r8) (7.21)

The SUMT iteration involves updating the multipliers and出einitial design vector and calling白eunconstrained minimizer again. In出巴algorithmit is assumed that山eDFP method is used (although any method from Chapter 6 can be used， BFOS is r巴commended).

Algorlthm: Exterlor Penalty Function (EP，丹Method伺又2)Step 1. Choose XI， N. (no. of SUMT iterations)， Nu (no. ofDFP iterations)

E;'S (for convergence and stopping)

r~， r! (initial penalty multipliers) Ch. c/I (scaling value for multipliers)

q = 1 (SUMT iteration counter)

Step 2. Call DFP to minimize F(Xq， r%， r~)

OutpUl: xq. Step 3. Convergence for EPF

If hk = 0， for k = 1， 2. . . . ， 1;

If gj:5 O. for j = 1. 2. . . . • m; If all side constraints are satisfied

Then Converged， Stop

Slopping Criteria:

d.F = FQ - F 0-" dX = Xq* -X(q-I)* q • q--I

If (d.Fl :5 EI: Stop (function not changing) Else If d.XT d.X :5 EI: Stop (design not changing)

日self q = N.: SlOp (maximum iterations reached)

Continue

q←q+l

rg ← r~'Ch; r:←r1・Cg

xq←xq. 00 to Step 2

潤1

7.2 INDIRECT METHODS FOR CONSTRAINED OPTIMIZATION 273

The EPF is very sensitive to the starting value of the multipliers and to th巴scalingfactors as well. Different problems respond favorably to different values of the multipli巴rs.It is recommended that the initial values of the multipliers be chosen as the ratio of the objective funclIon to the corresponding tenn in山 penaltyfunction a1 the initial design. This ensures出atboth the objective function and the constraints are equally important in deteロniningthe changes in the design for白巴 succeeding

lteration. One reason for the tenn Exterior Penalty is that at the end of each SUMT iteration

出巴 designwill be il砕asible(until the solution is obtained). This implies that出cmethod detennines design values白紙 areapproaching白efeasible region from the outside. This is a serious drawback if the method fails p陀 maturely，as it will often do. Thc information generated so far is valueless as the designs werc never feasible. As seen in the example below. the EPF severely increases出enonlinearity of the problem creating conditions for the me出odto fail. Il is expected白紙 theincrease in nonlinearity is balanced by a c¥oser starting value for the design， as each SUMT iteration starts c¥oser to the solution than出cprevious one.

ln the following the EPF is applied to Exarnple 7.1 through a series of calculations rather出anthrough the translation of山ealgorithm into MATlAB code. There are a couple of changes with respect to Algorithm (A7.1). To resolve the penalty function with respect to白einequality constraint， the constraint is assumed to always be in violation 80出at出ereturn from the mllx function is白econstraint function itself. This wi1l drive the inequality constraint to be active which we know to be true for this example. Numerical implementation as outlined in the algorithm should allow detennination of inactive constraints. Instead of numerical implementation of the unconstrained problem， an analytical solution is determined using MATLAB symbolic computation. Example 7.1 is reproduced for convenience.

Minimize f(XjoX2): x1-2xTx2+xT+XIX~-2xl +4 (7.9

Subjectω: h(x(，X2): xT+x~-2=0 (7.lOa)

g(x(， x沙 O.25xr+ O万x~-1 :50 (7.LOb)

o :5 Xl :5 5; 0 :5 X2 :5 5 (7.l0c)

Figure 7.2 is thc contour plot of the transformed unconstraincd function for values of rh = 1 andら=1. The increase in nonlinearity is rcadi1y appar宅nt.Figure 7.3 is the plot for rh = 5 and rg = 5. Handling山esefunctions numerical1y even with the BFOS wil1 be a considerable cha11enge. Both the figures suggest s巴veralpoints出atsatisfy first order conditions. Their closeness m池田itdifficult for any numerical technique to find the optimum.lt is clear白紙theEPF severely increases the nonlinearity ofthe problem. Sec7_2_1-plolm contains the code伽 tgenerates the plot. Since the code uses symbolic manipulation. actually drawing tlle plot takes time. Evaluating the data Dumerically will make a big difference.

274

制

民

4卜

NUMERICAL TECHNIQUES FOR CONSTRAINED OPTIMIZAnON

。

ーャ 7九..;..;..:

.. ・--暗唱、、

x t

z

..ー曹、 .'"

3 4

Flgure 7.2 E刈阿orPenalty Funcllon Melhod: Example 7.1 (n. = 1， rg = 1).

4

5

Appllcation 0' EPF A句orlthm(A7.2): The corresponding unconstrained func-tion for白isproblem is

F(XI .X2• 九. 'g)=(x~-2x1X2+xf+X1X~-2xl +4)+

日(x~ + xi -2)2 + '8(0お xf+O万三一1)2 (7.22)

Sec7 _2_1_calc.m is an m-fiIe that will calculate出esoh瓜onand the values of白e

function. for Example 7.1. for a predetermined set of values of the penalty multipliers.

It requires two inputs台。mthe user at different stages. The first input is the values for

白emultipliers for which出esolution should be obtained.百leIist of solution (there are nine for this problem) is displayed in由巳 Commandwindow. The user finds白e

solution that satisfies the side constraints (usually one) which must be entered ut白e

prompt (note: it must be entered as a vector). The program then prints out白evalues

of山evarious functions involved in白eexample. The following is posted from the

Command window for both of the penalty multiplier百setto 25.

持 Sec72 1 calc

enter value for rh [default 1] 25

3ヨ

ヨl道

N M

7.2 INDIRECT METHODS FOR CONSTRAINED OPTIMIZATION

5

$everc nonllnearlty wlth many locol mlnlmums

41

3・ ~ーー-cτ ， ..

x t

Flgure 7.3 Exterior Penalty Function Method: Example 7.1 (n. = 5. fg = 5).

enter value for rg [default 1] 25

ans -1. 4125 -0.1015

-0.8692 1.0980 -0.6263 -1.2107

-0.0278 -1. 3266 -0.0124 1.3267 -0.0090 -0.0000

0.7893 -1.1267 0.9775 1.0165 1.4184 -0.1318

Input the design vector chosen for evaluation

[0.9775 1. 01651 The design vector [ 0.9775 1.0165 1

objective function: 2.9810

equality constraint: ー0.0112inequality constraint: 0.0138

275

5

277

Here ).. is the multiplier vector tied to the珂ualitycons町aints，p is the multiplier vector associated wilh the inequality constraints， and rh and rs are the penalty multipliers used similar to lhe EPF method. F is solved as an unconstrained function for predetermined values of λp， r"， and rg. Therefore， the solution for each SUMT iteration is

7.2 INDIRECT METHODS FOR CONSTRAINED OPTIMIZATION NUMERICAL TECHNIOUES FOR CONSTRAINED OPTIMIZATION

In the above run， the equality and the inequality ∞nstr包ntar芭 notsatisfied (箇

expected). Table 7.1 documents the results of the application of白eEPF method to

Example 7.1 through Sec7 _2_1_calc.m.百lescaling factor is 5 for both multipliers.

A glance al Table 7.1 clearly i1Iustrates the characteristics of the EPF method . As the values of the multipliers inc陀 ase:

276

X.=X・(λp，rh， rg) . The design approaches出eoptimal value.

• The constraint violations decrease.

• The solution is being approached from oulside the feasible region. At山eend of the SUMT ileration lhe values of the multipliers and p巴naltyconstants are updated. The latter are usually geometrically scaled but unlike EPF do not have to

be合ivento∞ for convergence.

Algorlthm: Augmented Lsgrange Mu附ipller伺LM)Method伺7.3)Step 1. Choose X1， N. (no. of SUMT iterations)，

Nu (no. of DFP i~巴rations)

E匂j'S叫(fi“or刊c∞o叩nv刊巴r唱ge叩nc印eand stopping) rぺ;. r4; (Oi n凶副itia凶a1pe叩na叫a叫Ityml剛II削u凶llti匂i切Pμli託e隠

Ch， cg (sca1ing value for multipliers)

r~O"， r~阻 (maximum value for multiplier百)

ν，.JJI (initial multiplier vectors)

q = 1 (SUMT iteration counter)

Step 2. Ca11 DFP to minimize F(Xq， )..q， p"， rZ， r:) Output: xq.

Step 3. Convergence for ALM

If hk = 0， for k = 1， 2. . . . ， 1;

lfgj:三0，for j = 1. 2， . • . ， m;

(百島>Oforめ=0)

(IfVf+ kλ'kVhk +却~Vめ =0)If all sid巴constraintsare satisfiω


Stopping Criteria:

M'=Fq-Fq-1・aJ(= xq* -X<q-I)・

lf (M')2 S EJ: Stop (function not changing)

日seIf aJ(T d.X S EI: Stop (design not changing)

Else If q = N.: Stop (maximum iterations reached)

An important note日間dsto be mad巴.百leanalytical computation using symbolic computation appear吋伺sy.The same cannot be said for the numerical computation. especia1ly looking at Figure 7.3.

Augmented Lagrange Multiplier (ALM) Method

This is the most robust of白epena1ty function methods. More importantly it a1so provides information on白eLagrange multipliers at山esolution. This is achieved by

not solving for the multipliers but merely updating them during successive SUMT iterations [2，4). It overcom巴smany of the difficulties associated wi曲 thepena1ty function formulation without any significant overhend.

7.2.2

Trans如rmatlonto the Unconstralned Problem: The genera1 optimization

problem (7.1)ー(7.4)is凶 lsform吋 ωin山間山odof Lagrange multipliers

(7.23)

(7.4)

Minimize

府川 r8): f(X)やdX)zdトト∞剖)+トhk吋鳥トト(X)剖)

Table 7.1

i = 1. 2..... n

Exlerior Penally Function Melhod: Example 7.1

xls巧Sx，!

Continue

q←q+l

日←日+2rh h(X"O)

pq←pq +2 rll (max也(xq*).-pqI2r

8])

r1.←rrCh; r:←rfCg

s

0.0227 0.0295 0.0138 0.0037

7.5097e・004

ー0.0687-0.0302 -0.0112 -0.∞27

-5.9775e-004

f

2.9578 2.9651 2.9810 2.99判2.9988

X2

1.0391 1.0364 1.0165 1.0044 1.0009

X1

0.9228 0.9464 0.9775 0.9942 0.9988

r 8

1.0 5.0

25.0 125 625

rh

1.0 5.0 25.0 125 625

Iteration

'aq，-弓JA--εJ


xq←xq*

00 to Step 2

The ALM algori出m(A7.3) is available as a stand-alone program in ALM.m.百leprogram is long which is not unexpected. These a1gorithrns are not凶vial.To translate algorithm (A7.3) into working code is a佐官mendousaccomplishment， particularly if yourexpo制限toMATI.AB occurred through出isbook. The reader should walk through the code line by line.百 eorganization and structure are kept simple and straightforwal'd. Conunenls hav巴 beenliberally used as an cncouragement to underst.1nd the code.

ALM.m: The code ga由ersinput from the user through several prompts in the Conunand window. The prompts and由巳 finaliteration ar芭 copiedbelow. The significant features of the code are:

• The starting design vector is the first input to山己progr創n.The number of design variables is automatically extracted from this information.

・百lelower and upper bounds on血巴 designv回ables(side cons紅白nts)釘巴

obtained出roughuser prompts (though曲eyare not used in山eprograrn).

• Number of equality constraints and corresponding initial multipliers (λ)ぽ巴

promptcd.

• Number of inequaJity叩 nstraintsand corresponding initial multipliers (日)脱promptcd.

• Values at cach SUMT itcration加古 printcdin the Command window. Wamings generated by MATLAB紅巴 switchedoff.

• The DFP.m is used for uncons回 inedoptimization. This program uses

GoldSect“ion一nVa町r• τ百oeprogram ex却Pε氏ct匂sthe following 臼e伐stωobe available i加n出ep伊a白止:

Objective function Ofun.m

Equality constraints Hfun.m

Inequality constraints Gfun.m

Unconstrained functIon FALM.m

• The program uses global statements to communicate multiplier values.

・Theinitial penalty multipliers are computcd automatically.

• Several parameters are coded into出eprogram. especiaIly those needed for the golden section and the upper bound calculation. These can be changed by出巴

us巴r.if necessary.

Application to Example 7.1: Thc following represents the initial prompts secn in the Command window and the values for thc starting iterations. Other iterations are simi1arly displayed. Also inc1uded below is a consolidated printing

7.2 INDIRECT METHODS FOR CONSTRAINED OPTIMIZATION 279

ofvalues for a11 ofthe SUMT iterations at the termination ofthe program. The initial

design vector is [32 ]. Initial λ= 1 and ~ = 1.

持 ALM

1nput the starting design vector This is mandatory as there is no default vector setup The length of your vector indicates the number of

unknowns (n) Please enter it now and hit return for example [ 1 2

3 ...]

[3 2]

誼t

The initial design vector:

3 2

1nput the minimum values for the design vector. . These are input as a vector. The default values are 3 * start value below the start value unless it is zero. 1n that case it is -5: [0 0]

The minimum values for the design variables are 0 0

-1!ci--

1nput the maximum values for the design vector. These are input as a vector. The default values are

3 * start value above the start value. 1f start value is 0 then it is +5:

[5 51

The maximum values for the design variables are 5 5

Number of equality constraints [0] 1

1nitial values for lambda 1

Number of inequality constraints [0] 1

1nitial values for beta 1

ALM iteration number: 0 Design Vector (X) 3 2 Objective function 64 Square Error in constraints(h， g) 1.2100e+002

1.8063e+001 Lagrange Multipliers (lambda beta): 1 1 Penalty Multipliers (rh rg): 5.2893e-001 3.5433e+000

280 NUMEAICAL TECHNIQUES FOA CONSTRAINED OPTIMIZATION

(other iteration omitted)

X not changing O.OOOE+OOO reached in 5 iterations

The values for x and f and 9 and h are

3.0000e+000 2. OOOOe+OOO 6. 4000e+001 1.8063e+001 1.2100e+002

7.9603e-001 7.1223e-001 2.9443e+000 0 7.379ge-001

9.9937e-001 1.0092e+000 2.9995e+000 1.8223e-004 2.9434e-004

9.9607e-001 1.0013e+000 2.9962e+000 0 2.731ge-005

9.9940e-001 1.0002e+000 2. 9994e+000 2. 5872e-011 6.2772e-007

9.9940e-001 1.0002e+000 2. 9994e+000 2. 5872e-011 6.2772e-007

The values for lamda and beta are

1.0000e+000 1.0000e+000 -9.0876e-001 0

-1. 6493e・0019.5663e-001

9.1192e-002 9.5616e-001

ー7.6430e-002 9.9221e-001

Th巴programtook five iterations (lifth was not necessary) to find the solution as:

x;=O.卯94;x; = 1.0002; r = 2.9994; h. = 0.0000; g. = 0.0007 (7.24)

which is very c10se to the actual solution. The values for山emultipliers釘巴

A.' = -0.0763; s" = 0.9922 (7.25)

This satisfies the KT conditions. However， in Section 7.1.2 the values of the

multipliers， obtained using symbolic computation， are quite different. It is difficult to explain the discrepancy except to say白紙 thesymbolic calculation failed to discover

this particular solution which is indeed possible.百levalues in (7.25) ar芭 mo問acceptable山anthose in Section 7.1.2 considering白eKT conditions.

TheALMme山od，as demonstraled above， is釦 effecliveand useful method in sDite of belonging to the class of indirect me山ods.It has several significant advantages over the other penally function formulations [2]. Som巴ofthemare lisled below: ー

• The method is not sensitive to the initial penalty multipliers or their scaling strategy.

・Themethod does not陀 quire山epenalty multipliers to be driven to extreme values 10 ωtablish convergence.

・ The equality and active constraint can be satisfied prl田 isely.

3週7.3 DIAECT METHODS FOA CONSTRAINED OPTIMIZATION 281

. The starting design can be feasible or infeasible.

• The initial choices for th巴multipliers田 nbe relaxed.

・Thereis only a modest increase in the calculations.

• At convergence the Lagrange multipliers wi1l be driven 10 tlleir optimum values. This aJlows verification of KT conditions through the sign of the multipliers

associated wi白血eactive inequality cons位aints.

7.3 DIRECT METHODS FOR CONSTRAINED OPTlMIZATlON

:剖1

The direct methods inc1ude both the objective叩 d出.econstr凶ntsto search for the

optimal solution. While白emethods do not involve conversion to a different c1ass of problems (like Section 7.2)， most of them are b鎚 edon linearization of the funclions about the currenl desi伊 poinl.Linearization is based on expansion of the function about the current variable values using the Taylor seriω(Chapter 4).

Unesrlzatlon: The Taylor series for a lWo.variable expanded function j{x， y)，

quadratically about ilie current point (xp' Yp) is expressed as:

的 1創刊)= f(xf" y/，) -1-[ま|J13|{J+!I割削+2蒜|叫+劉州側)L _.. I (x，.JI，) ----" I (xpJ'，) -" I (弘司ト J

r司調咽J宮高崎iy一cJuud町三

If白edisplacements are organized ωa column v郎防 [ax 8y]T.山 expansionin

Equation (7.26) can be expressed in a condensed mannerω

f汽(円山)ド吋=ザ吻f

L片[釘 A勾制y刈]T[凹H(仇;Xn.Yn凡υIJ]川]I 釘刊(7.27)ι ，. ，. IsYI 、，

For It variables. with Xp the current point and 8x the displacement vector，

f仲必)= f(Xp) + Vflぽp)T8X+tdXTH叩成問)

Equation (7.28) c佃 bewrilten in terms of白edifference in function valuesぉ

ザ=f(Xp+尚一f(Xp)= Vf(Xp)T dX + t dXT町Xp)組問)

df=Of+o2f


where of = VjT aX is tenned白巴Jirstvariation. o2jis the second varialion and is given by白es田 ondtenn in the above叫uation.In linearization of the functionfabout the

current value of design Xp' only the fi附 variationis used. Th巴neighboring叫 ueof

the function can be expressed as

l(Xp) = f(Xp) + V f(Xp) T aX (7.30)

All of the functions加山eproblem c加 belinearized simil釘 ly.It is essentiaI to understand山 differencebetweenf(X) andl(Xp)' This is ilIustrated in Figure 7.4 using the objective function ofExnmple 7.1 expanded about the current designx， = 3， x2=2. Th巴curvedIinesf(X) are the contours of山eoriginaI function. The s位Bightlines J(Xp) are the contours of the linearized function (lines of constant vaIue of the function) obtained through the following:

OriginaI印刷ion: f(x). X2): x1-2 xrX2 + xi + X，~ -2x1 + 4

Linearized function: ](!J.XI' !J.X2) =ω+92!J.XI-6!J.X2

where the coefficients in白己 secondexpression includes the evaluation of the

function and gradient atx， = 3， X2 = 2 and substituted in E伊 ation(7.30). In Figure

7.4 several contours are shown although those of int巴restare in the neighborhood

of出ecurrent design. Any nonlinear function can be expanded in the same manner. If another point was chosen. then山eslopes 01" Lhe lines would be different.

The quadratic expansion of the function can be obtained by using Equ凶on(7.28).

百leexpanded curves will be nonlinear (quadr甜 c).百leyappearぉ ellipsωinFigw芭7.5

where severaI contours of曲e0凶ectivefunction of Example 7.1蹴 expanded

quadrati回lIya加utXI = 3， X2 = 2. It is important to附 ognizethat山econtour冨wouldbe

different if another point we陀 chosenfor expansion.

Four direct methods ar芭 discussedThe first is SequentiaI Linear Prograrnming (SLP)， where the solution is obtained by successively solving the corresponding

linearized optimization problem. The s民 ond，SequentiaI Quadratic Programming (SQP)， uses山equadratic expansion for出eobjective function. Like出eSUMT，出e

current solution provides山estarting vaIues for the next iteration. The third.出e

GeneraIized Reduced Gradient Method (GRG). develops a sophisticated s朗氏hdirection and follows it with an elaborate one-dimensional process. The fourth. Lhe

Sequentia1 Gradient Restoration Algorithm (SGRA)， uses a two-cycle approach

working on feasibility and optimality a1temately to find the optimum.百lereare

severa1 other methods due to many researchers but山eydiffer from由oselisted in

small details. To k田 pthe length of this chapter reasonable.出eyhave not been

incJuded. With由ecode叩 dother procedures available from this section. it should not

bedif白cultto program th巴additionaltechniques出atare worth exploring for particular

cJasses of problems.

Except for special classes of problems in the following two chaplers.白ischapter

is the principal reason for the book. It provides a resolution of the complete

optimization problem as posωlated. For a long time now there have not been any new

or greatly improved numerical techniquω.百lereis a lot of work on global optimization.

7.3 DIRECT METHODS FOR CONSTRAINED OPTIMIZATION 283

N M

5

4

3

t222dt民主4Af

F d

-1 x t

F1gure 7.4 lIIustratlon olllnearizatlon.

n Current interest in traditional desi伊 opti出回tion(白ischap町)， is directed toward

actual design appucadons and migration fmm themainfmme and UNIX Environment

to白ePC (like出isbook)・Forcompleteness. the standard fonnat of the NLP is

reproduced here for convenience:

Minimize !(x1o X2. • • • • x，，) (7.1)

Subjectω: ht<X"X2.・・・，XII) = o. k = 1， 2. . . . • 1 (7.2)

め~XI ・ X2.... ，ん)孟O. j=1，2....，m (7.3)

XlS:XiS:Xr. i=1.2....，n (7.4)

In vector notation

Minimize f(X). 医]"(7.5)

Subj田 tto: [h(X)]，=O (7.6)


同同

5国百子宮

4

3ト J，，， 2"'"

-2 ・2

I ，，，

，

，，十，

-1

x v

2 3

FJgure 7.5 lIIustration 01 quadratlc 9xpanslon.

[g(X)]m S 0

X10wSXSXup

4

(7.7)

(7.8)

Example 7.1 will be used to ilIustrate the various algorithms.

Minimize f(x.，xi): x1-2xix2+xi+xlrz-2xl+4 (7.9)

Subjectto: h(XI， x:z): xi + rz -2 = 0 (7.10的

g(X..X2): 0.2sxi+0.75~-1 SO (7.10b)

OSXI ::;5; OSX2SS (7.lOc)

7.3.1 Sequentlal Linear Programmlng (SLP)

5

In the SLP [5] all of the functions are expanded linearly. If X; is considered山ecurrentdesign v回 tor，出en出elinearized optimal problem can be set up as

7.3 DIRECT METHODS FOR CONSTRAINED OPTIMIZATlON 285

Minimize: l(aX) = f(X;) + Vf(X;) T aX (7.31)

Su尚氏tlo: 月以aX): "k(X;) + vhI (X;)aX = 0; k = 1，2， . . . ，1 (7.32)

あ(AX): め(X;)+ V8J{X;)T AX S 0; j = 1，2.. .. ，m (7.33)

Ax!ow壬ax;S ax)IP; i = 1，2， .. . ，n (7.34)

Equations (7.31)ー(7.34)represenl a Linear Programming (LP) problem (Chapter 3). All of the functions in Equations (7.31)一(7.34)，except for aX. have numerical values after substituting a numerical vector for X;. Assuming an LP program code is available， it回 nbe called repeatedly after the design is updated as

X"， =X，+AX 件l-~.i

In order to include a search direction and stcpsize calculation，出巴企Xin Equations (7.31)ー(7.33)is considered as a solution for S (a search direction at the currenl design). Stepsize computation slra飽gyis no longer simple. If出巴reare violated constraints， a1pha attempts to reduce白isviolation. If白ecurrent solution is feasible，出estepsize will attempt to reduce the 白nctionwithout causing the constraInts to be excessively violated. Such strategies are largely implemented at the discretion of由巳

algorithm or code developer.

A匂orlthm:Sequentlal Llnear Programmlng (SLP) (A又4)Step 1. Choose X'， Ns (no. of iterations)，


q = 1 (iteration counler)

Step 2. Call LP to optimize (7.31)一(7.34)

Output: S

Use a constr叫nedα・ calculation (X; = xQ) AX=α.. S

Xq+I=Xq+AX

Step 3. Convergence for SLP

If hk = 0， for k = 1， 2， . . . ， 1;

If gJ S 0， for j = 1. 2， . . . ， m;

[f all side constraints are satisfied

If KT conditions satisfied


Stopping Criteria:

AX=xq+l-xq

If AXT AX S EI: Stop (design not changing)

If q = N.: SIOp (maximum iterations児 ached)

286 NUMERICAL TECHNIQUES FOR CONSTRAINEO OPTIMIZATION

Continue

q←q+l

00 to Step 2

The SLP is applied 10 Example 7.1 below. In Chapter 3 no explicit program for solving 白eLP program was developed although a command-by-command walk白roughwas introduced. A program for applying白eSimplexme白od，pslmplex.m is included on 白ewebsite (without any comment or discussion). Using instructions from Chapter 3， adding somc program control， and organizing some bookkeeping can provide a MATLAB program for LP problems. Altemately， M ... 百.AB'Slinprog can be used in a loop to help lranslate出eSLP algorithm to code. In出issec“on曲elinearized subproblcm in each iteration is graphicalIy presented. The area of interest can bc zoomed and the solution read from the plot. This is treated as dX (instead of using it as S). The design is updated and another plot can be obtained by running the m・filefor a new value of the design vector. Sec7 _3_1-p1ot.m assists in displaying the Iinearized functions from which solution can be obtained. The equality constraint is 出egr田 nline.百lered is inequalily constr通int百ledesign vector is回加putto白eprogram. Th巴∞deuses symbolic calculation 10 solve and display the subproblem.

Appllcatlon of SLP sωP 1. Xl = (32) Step 2. Linearized subproblem r . ，

Minimize 1(dX) = 64 + [92 -6] I ~XI I IsX21

釦ゆct肱 II(dX):11 + [6 4] I仏1=0I !lX21

g(dX): 4.25 + [1.5 311内IsoI !lX21

FromFigure7.6(zoomed)thesolutionis~ =axl =ー1.335，&2 = ax2 = -O.75.For thenext iteration X2 = [ 1.665， 1.25]. Repeat Step 2.

Step 2. Lin巴arizedsubproblem

Minimi吋ほ)= 6.7985 + [13凹05-1.382] 1 ~XII I LU21

S雌 ω:百(必):23347+[333251lh|=0IsX21

lax.l-g(dX川臼9+附 251.8751/的ISO

酒

温

調

掴7.3 OIRECT METHODS FOR CONSTRAINED OPTIMIZATION 287

5

4

-4 -3 -2 4 5 2 3 o OX

1

Flgure 7.6 Example 7.1 SLP ileration 1.

-

-i3iifzj-

From Figure 7.7 (zoomed) the solution is axl =ー0.54，ax2 =ー0.23.For the next iteration X3 = [1.125， 1.02]. Repeat Step 2.

Step 2. Linearized subproblem

Minim回 1(.瓜)= 3.206 + [2.3957 -0.2363] Iふ|l品 "'21

釦 bject10:五(広):0.3060 + [2.25 2.04] 1的1=0lいι"'2引l

E削削叫凶附附):川:氾0ω0…6From Figure 7.8 (z初oomedの)the solution i凶sdx川(=一0.1ロ2笥5，dx2 =ー0.02.For the next

iteration X4 = [ 1.0 1.0 ]. Since this is the known solution， further iterations釘芭

stopped. 百leSLP took three iterations t.o find the solution. This is very impressive. It is true

only for th.ose cases where the number .of equality plus active incquality constraints is

羽

詰

lIi

288 NUMEAICAL TECHNIQUES FOR CONSTRAINED OPTIMIZATION

5

4

3

N

>< 0 <0

-2

-3

4

-5 ・5

Flgure 7.7 Example 7.1 SLP lIerallon 2.

4 5

lhe same as出enumber of design variables-a full comp[ement. The main

disadvantage of lhe melhod appe釘sif白ereis no full complement. In lhat case出巴side

cons位置ints(7.34) are critical to出edetennination of lhe solution as山eydevelop into

active constraints. In lhese situations lhe limits on lhe side constraints創ecalled lhe

move limits. ln回 seof active side constrninls Ihe move limils establish the values of

lhe design changes. If lhe design changes are consider吋 aslhe search di陀 ction(s回

algorilhm)，山enbo曲 vaJueand direction are a仔巴cted.For lhese changes to be small

as lhe solution is approached，出emove limits have to be adjusted (lowered) with every

iteration.百lestrntegy for血isadjustrnent may influence白esolution.

Example 7.1 wilhout the inequality constraint， but wilh白eequality constraint

provides an illus位置lionof the move limits. Figu問 7.9represents lhe lin回 rization

about lhe point [ 3， 2 J. The move limits in lhe figu陀 (box)represent lhe side constraint

limits

-2~ Lttl 三 2; -2 ~ÂA1 S2

lt is clear from Figure 7.9 Ihat wilhoul Ihe move limits thc solution is unbounded，

which is not very helpful. Using lhe solution obtained in lhe figure， lhe next iteration

き自1

7.3 DIAECT METHODS FOA CONSTRAINED OPTIMlZATION 289

5

4

3

封同。“3

ー1

-2

-3

-4

--55 L -4 -3 -2 -1 。 2 3 4 5 SX1

Flgure 7.8 Example 7.1 SLP lIeration 3.

C組 beexecuted. Once again lhe move lirnits will influence出esolution. If the move limits are left at lhe same value， the solution will always be on lhe squぉewith no possibility of convergence. These lirnits musl be lowered with回 chiteration. Usually

they are done geometrically lhrough a scaling faclor.百lisimplies lhe convergence will depend on lhe stralegy for changing lhe move limits， not an appealing situation. The SLP is not one of lhe popular melhods. 111e move lirnits are avoided wi由lhenext

me出od.

7.3.2 Sequentlal Quadratlc Programmlng (SQP)

The lack of robustness in由eSLPdueto出eneed for move lirnits can be countel吋 by

including an element of nonlin回 rityin白eproblem. The陀 arcseveral ways lhis can be achieved. One way is 10 require lhal山巴 searchdirections (remember in acωa1 implementation sX is S) in Equations (7.31 )ー(7.34)be limited by the unit circle which can be described白巴constrainl:

STSS: 1 (7.35)

290 NUMEAICAL TECHNICUES FOA CONSTAAINED OPTlMIZATION

5

3

2

宏、.z o

-1

-2

-3

-4

-5; -4 -3 -2 o

6x唱

Flgu問 7.9 lIIustratlon of move IImlts.

百】巴 stepsizedetermination should then account for the actu叫changein the design

vector. The problem expressed by Eq岡山田 (7.31)一(7.35) is a quadmtic

programming (QP) subproblem. An itemtive solution of this subproblem c血 beone

form of恥 SQPme出od.This 脱出odis popular and is used widely. While the QP

subprogram is a simple nonlinear problem with linear cons位置ints.it is still a

multivariablc nonlinear problem. Fortunately， QP problems appear in many

disciplines and there are efficient procedures and algorithms to address these problerns

[6-10]. One of th巴semethods is an extension of the Simplex method of Linear Programming.

百lesolution to白eQP is well documented. It would rl叫uiremuch discussion and

therefore is not introduced hcre. Readers are referred to the various references. This

is also白eprimary reason why the QP method is available in the Optimization

Toolbox in MATLAs. The formal methods for QP problems are based on efficiency in

the various stages of the implementation. An inefficient but successful solution 10 the

problem can be obtained by using Ihe ALM of the previous section to solve the QP

subproblem. After all. computing resources are not a constraint today. In lieu of formal

discussion. an intuitive understanding of白enumerical method is attemp低din this

book. The translation of SQP into code is not trivial and the method is usually the basis

of commercial software. For example， SQP is the only algorithm for constrained

7.3 DIAECT METHODS FOA CONSTAAINED OPTlMIZATION 291

optimum白紙 MATLABimplements (as of this writing). Having access to the

Optimization Toolbox allows NLP to be solved using SQP. A detailed discussion of the solution technique is also available [11].

百leQP subproblem employed in this subsection is based on expanding thc objective function quadratically about the current design.百leconstraints are

expanded linearly as in SLP [8].百 isis called sequential quadratic programming (SQP).

Minimω 1(d.X) =f'叫)+ Vf(Xir d.X +伊TV2fl附 dX 間)

Subj田 lto: 九(以):hk(Xj) + VhI (X，)ほ=0; k= 1.2.....1 (7.32)

~J{ð.X): gj(Xj) + VgJ{Xj)T d.X S 0; j= 1.2..... m (7.33)

sx}ow S sxj S ðx~P; i = 1.2， .・..n (7.34)

In Equation (7.36) Vす(X，)is the Hessian matrix. In actual implementation the real

Hessian will not be used. Instead a metric開]白紙 isupda飽dwi出 eachiteration is used.百世sis based on出巴 successof the Variable Me凶cMethods (V恥仏.f)of the

previous chapter. Several researchers [6.9] have shown出鉱山巴 BFGSupdate for the Hes8ian provides an e仔icientimplementation of the SQP method. This QP is well posed and convex and should yield a solution. Solution to this subproblem is at由巳

heart of山eSQP method. A considerable amount of research has been invested in developing efficient techniques to handle 出is subproblem. In a formal implementation of恥 SQP山d.Xin Equations (7.36)加 d(7.32)ー(7.34)musl be 回 placedby the search direction S. The QP for S al80 modifies the constraint equations

80 that a feasible direction can be found wi白respectto白ecurrent active constraints. Figure 7.10 illus岡鉱S白econcem about moving linearly to a point on an active con8佐aint.XO， the current design is on an active constraint. If the search direction SI follows出巴 linearizedfunction at XO (the tangent to the function at that point). any stepsize. along自己 E問見hdirection. however small， will cause the constraint to be violated. In order to determine a neighboring point that will sti¥l satisfy the constmints. a search direction slightly less th朗自etangent is used. This introduces a lack of consistency as the deviation from tangency becomes a matter of individual experience andpracti田 .Experiments have suggested白at90-95% of the tangent is a useful figure

although making it as close to 100% is recommended [2].τbe search direction finding QP subproblem is

Minimize 7(S) = f(X1) + Vf(Xj)T ST[H]S (7.37)

Subjeclω: 九(S):cMXi) + VhI (X，)S=O; k= 1.2.... ，1 (7.38)

あ(S):cgi<Xi) + V 8.メx，)'fS S 0; j = 1. 2ぃ..，m (7.39)

s}OWSSjSSr"; i=I.2.....n (7.40)

Rep伺cingHessian with BFGS Update: ln Eql鳩山n(7.37)， calculating the actual Hessian is strongly discouraged. Since the BFGS m巴出odconverges to the

Hessian， the BFGS metric is considered to be an excel1ent replacement of the Hessi佃[9]. Theme凶cis based on the Lagrangian of出efunction [12]:

Calculating the Mulflptters: The popularity of the SQP m白血odis that it attempts to update出eHessian based on the Lagrangian of the problem.

Calculating the multipliers is necessary to verify the Kuhn-Tucker conditions at

th巴solution.Calculating the multipliers， for the current value of the design vector， is necessary for additional operations， including setting up the descent function in Equation (7.43)， as well as updating the metric replacement for the Hessian (follows next). The calculation uses th巴FOCbased on the Lagrangian. The values

of the gradients of the various functions are computed at the current design. The

multipliers 紅白 thenobtained as a solution to a set of linear equations. The

solutions to QP and SQP problems usually take advantage of th巴activeconstraint set. This includes all equality constraints and only those inequality constraints that are active. This implies that SQP iterations start at a feasible point. The multipliers corresponding to inactive constraints are set to zero. The remaining

multipliers can be established considering a set of an appropriate number of linear

independent equations from Equation (7.12):

Equation (7.42) or (7.43) can be used to obtain白evalue g stepsizeα. The function in

Equation (7.43) requires an important observation. Should the Lagrange multipliers

be calculated for each value ofα? Imagine using the golden section to solve Equation (7.43) with血emultipliers calculated企omthe sa出factionof白eKubn-Tucker

conditions for each new value of X. Several updating strategies for由巳 multipliersin

Equation (7.43) can be found in由elist巴dreferenc巴s.A simple one is to hold the values

of the multipiers constant 創出eprevious iteration value and change them at白eend

of the iteration.

(7.43)

(7.41)

。.42)

293

i= 1，2，.. .， n

(7.12)

7.3 DIRECTMεTHODS FOR CONSTRAINED OPTIMIZATION

nu x

め蹴m

・・J

Rur

mでム一円

+

ぽ，Jr

内

A

J

す血制

+

X

FJ

=

はAV

内，.nu

mA 句協m

mTL-H

re +

内

4

x

ικ tn

ーす臼制

，， +

.r ''E、，J 一一X

AV

eJF eJf " dh， . oh， • n e}g 1. . n e}g 一=止+λ 」+ー・+L-+PI」+・・・ +~n ー!!!.=O;axi axiftaxitaxif l axiiaaxi

xj=x;'聞 1+αS

Minimize:

Minimize:

The side constraints are not necessary総出isis a well-posed problem.官lefactor c is the adjustment for moving tangent to the constraint.百lesuggested value for c is 0.9-0.95. For the inequality constraint，出evalue of c is 1 if the constraint is satisfied (should be able to move tangent to the constraint).

Stepsize Calculation: On回出esearch direction has been determined， it is necess紅 Yto calculate the stepsize. Since the methods of出 schapter have seen an enormous increas巴inthe number of calculations， the stepsize calculations are based on s加lUltaneouslydecl'回sing血eobj田 tiveas well as improving出巴 constraintsatisfaction. In order to accomplish this goal a suitable function must be developed and 山estepsize determination based on minimizing this unconstrained function. Several

authors refer to it as the descent fllnction. Th巴descriptionof this function varies from

researcher to researcher. Some define白isas the Lagrangian function in whicb case 曲巴 Lagrangemultipliers must be determined. Many others include only the

constraints with a penalty multiplier and adopt山巴 EPFform for白巴 function.Still

others include only the maximum violated constraint. Two forms of the function az芭

¥isted below:

4 3

Movlng linear to an active constraint.

NUMERICAL TECHN1QUES FO円CONSTRA1NEDOPT1MlZATION

2

Flgure7.10

><" 1

-1

292

294 NUMERICAL TECHNIOUES FOR CONSTRAlNED OPTIMIZATION

p=Vf(X判)+LAk Vhk(X伊 1)+エ sjVCj(X'計')

1<=1 j=IVjeJ

-Vf(Xq)+ヱ入'kV hk{X~ + L sj V gixq) (7.44)

1<=1 j=lltjeJ

y=Xq+I_Xq

[Hq+l] =間J+単一耳目阻p'y y・[Hq]y

j E J impIies the set of active inequality constraints. It is recommended白紙白巴me凶cbe kept positive definite as much as possible even if it is likely白atit wiIl nol be positive definite at the solution. One way of ensuring白isis to verify that

pTy>O

for each iteration.

A匂'orlthm:Sequential Quadratic Programmlng (SQ門伺7.勾Step 1. Choose X I ， 1叫(no.of iterations).

E/S (for convergence and stopping)

q = I (iteration counler)

Step 2. Call QP to optimize (7.37)一(7.40)(X1 = xq)

Output: S

Use a constrainedぜcalculation

AX=α" s Xq+1 =Xq+AX

Step 3. Convergence for SQP

Ifhk=O，fork= 1.2，....l;

立身~O， forj= 1.2，...， m;

If KT conditions are satisfied

百四nConverged. Stop

Stopping Criteria: AX=xq+l_xq

IfAXT AX ~ EI: Stop (design not changing) If q = Ns: Stop (m位回umiterations reached)

Continue

UpdateMe肘c[町

q←q+I nn rn Stpnヲ

l沼i

7.3 DIRECT METHODS FOR CONSTRAINED OPTIMIZATION 295

Application 01 SQP: Th巴 SQPtechnique is applied to Exarnple 7.1. There are several ways to accomplish this. The application can use graphical support as shown with the SLP (this is an exercise). The application can use the QP program from MATLAB (also an exercise).ln曲isbook白巴QPproblem is solved by applying曲eKTconditions-wbich is the feature ofthis subsection. Ati飽rall for出isspecific example 白efunctions are analytically defined and血eirstruc組問is simple. We can hamess the symbolic operations泊MATL油 tocalculate白ederivatives and白eactual Hessian. As in山ecase of SLP， the actual chang巴 intlX is computed [Equations (7.36). (7.32)ー(7.34)]rather than the search direction S for the QP subproblem.

Sec7 _3_2.m applies the SQP me出odsymbolically to Example 7.1. lt identifies the solution using出eLagrange method (Chapter 5). It promp凶 forthe current design vector. It solves the QP analytically applying the KT conditions and providing白eresults for the two cases. C邸 ea corresponds to s = 0 (g must be less由anzero). Case b corresponds to g = 0 (s > 0). The output from running the program is recorded below for several iterations in出isimplementation of SQP. Ana1ytical solution with user intervention is us叫.Sec7 _3_2.m must be run for each iteration. You should be able to copy and past巴desi伊 vectorin自己commandwindow.

Iteration 1:

Iteration 2:

XI =[3勾

M 町叶=イ[戸2lトい問岡=[宝 ?可l 1; Vh=ぺ寸[~Jg=叫叫叫4ωω2お5刊刊

d.xl =-1.0591; ~ =ー1.1613

五=0; g = -0.8225;

X1 = 1.9409; 毛=0.8387

X2 = [1.9409 0.8387]

州 3.123; サ=UZ;什 1222剣山 705; Vh=[悦 Jg=0.4693; η=11矧

The soIution to the QP problem is Case a:

d.xl=ー0.6186; d.x2=-O.0041

h=O; g=ー1.8281;

蕩f

296 NUMERICAl TECHNIQUES FOR CONSTRAINED OPTIMIZATION 7.3 DIRECT METHODS FOR CONSTRAINED OPTIMIZATION 297

茸1= 1.3222; 毛=0.7975

Iteration 3: X3 = [1.3222 0.7975)

f=4.2126; 町=ド立J問=似品 2281

=0 牧町=!?剣山0858; Vg刊;;;l

7.3.3 Generallzed Reduced Gradlent (GRG) Method

百1巴 GRGmethod is anothcr popular technique for constrained optimization.百leorigina1 method， the Reduced Gradient Method [13]， has seen severa1 different customizatiODS due to severa1 researchers [14). Like出巴 SQP白isis ao active set method-deals with active inequalities-hence the method iocIudes珂ualitycons回 intsonly. The in叫ualitycons回 ints釘 E位四sformedto equa1ity co田町aintsusiog a linear slack variable of the type used in LP problems.τne genera1 optimization problem is restructured as

The SOIUtiOD to the QP problem is Case a:

Iteration 4:

Minimize f(XI' X2， • • . ，Xn) (7.45)

dxl = -0.2685; dx2 = +0.2042 Subject to: ht(XI' x2'・・・，x，，) = 0， k = 1， 2， . . . ， 1 (7.46)

(7.47) h = 0; g = -0.0190;

8J(XIo X2' . . . ，x，，) + x，t+j = 0， j = 1，2， • . . ，m

X1 = 1.0537; ち =1.0018 xlsx;sxr i= 1，2，...， n (7.48)

Xn+j ~ 0 j = 1，2， • .. ，m (7.49) X4 = [1.0537 1.0018]

f= 3.0685; Vf=ド法~J昨同22Ul

…沼;vh=I2124;g=ー0問内=lTZlI!~

The total number of variables is n + m. The active !:et !:trntegy ao;sumes that it is possible ωdefine or choose n subset of (11 -1) independent variables合omthe set of (l + m) equality constraints. Once these varinbles are estabIished， lhe dependent variables (l + m) can be recovered through出巴consLraints.The number of independent variables is based on山coriginnl design varinblcs and the original equa1ity constraints of the problem. The fol1owing development is ndapted from Reference 2.

The set of the design variables [X] is partitioned into出eindependentset[Z]叩 ddependent set [11.

dx. =-0.0523; L1xz=-O.0017; ト 0.9275

五=0; g=O; 、，E・E

・-EJZ

Y

rE

・E

・-EEah

=

x

1. = 1.0014; 毛=1.α胸 ; The constraints are accumulated as

f=3.0014; h=O.∞27; g = 0.0006 H=[:]

The solution is a1most converged and furthcr iterations are left to the reader. The next iteration should produce va1ues for convcrgence.

The SQP here takes five iterations to converge. This was based on computing the actua1 Hessian. The SLP took four. The ALM took five. For Example 7.1出eSQPdoes not app回 rto have姐 ysignificaot advaotage， which is unexpected. The ALM slarts to appear qui飽 attractive.The SQP in Sec7 _3_2.m is implemented analytical1y based on KT conditions. It is very different than Algorithm (A 7.5) which rep陀 sentsa numerical impl巴mentation.

Re1ations (7.45)ー(7.49)can be recasl as

Minimize f(Z， Y) (7.50)

Subject 10: H(Z， Y) = 0

z!S z;Szf i= 1，2，...， n-l

(7.51)

(7.52)

298 NUMERICAL TECHNIQUES FOR CONSTAAINED OPTIMlZATION

A similar set of side constraints for Y can be expressed. 1n ac伽alimpl巴mentalionthe

inactive in叫ua¥ityconstraints are not inc1uded in白巴 problem.Also， the slack variables can b出 ngto the independent制.In GRG，山 lin削 zed01吋叫veis

minimiztd suhdect to thelinearizcd constraints about the cumnt design r.

Minimize ](l:J.z' l:J. Y) = f(Xi) + V zf(Xi? d.Z + V yf(Xi? l:J. Y (7.53)

Subject to:払(!:J.Z，l:J.Y) = H.(Xi) + VzH.(Xi)T d.Z+ V.;.叫(Xi)Tl:J.Y= 0 (7.54)

lf the GRG began at a feasible point.出eB(Xi)=0，出elin巴arizedconstraints are a¥so zero，出巴n恥 iblearizedequationsin問]detennine a set of linear equations in l:J.Z and

d.Z. Considering出巴vector[H] the linear equation can be writtenぉ

where

[A]d.Z+匝]l:J.y=o

r V~Hn r VvHT 1 I Y ZIIJ I I • ya&1 I

1 v~1i!" 1 1 Vν 耳|[A] = 1 -. -1; [B] = 1 " -1

・・・・・・

I Vz味mI I V)正札

(7.55)

From Eq凶tion(7.55) the change in山edepcndent variable l:J. Y can be expressed in

lerms of the changes in thc independent variables as

l:J.Y=ー[Brl[A] d.Z (7.56)

Substituting Equation (7.56) into Equation (7.53) reduces itωan unconstrained

probl巴min d.Z:

](必，)= f(Xi) + V zf(Xi)T d.Z -Vyf(Xi)T[Brl[A]d.Z

= f(Xi) + {v zf(XうT-VyflほうT[Brl[A])l:J.Z (7.57)

=バXう+{Vzf(Xi)ー([町l[Alfvdxt)「必= f(XI) + [GR]T l:J.Z

[GR] is出ereduced gradietJt of the functionj{X)・Itprovides the search direction. The

蹴ふizeαisfound by J1叩iring伽 t恥伽ng巴sinl:J.Y，∞J1J1e5ponding10恥 ch叫 es

in hz(αS)， determine a fl回 sibled叩 gn.This is implemented血roughNewton' s method as described below.Since thehnctions ME linear，them is no quadratic

'11111 7.3 DIRECT METHODS FOR CONSTRAlNED OPTIMIZATION 299

convergence associated wi白山ischoice. The iterations start with a feasible current design vector [Xi] = [Z汀 Y吋

Stepslze Computatlon: A旬。rlthm(GRG)Step 1. Find GR， S = -GR; Select a

Step2. q= 1

d.Z=αS; Z = Zi +!:J.Z;

l:J.yq=ー[BrI[A]d.Z

Step 3. yq+l = Yq + l:J. yq

x1+1=1 ZI Iyq叶|

If[H(X削)]= 0; Stop， Converged

日seq←q+1

l:J.Y9=凹r1[-H(Xi+I)]

GoTo Step 3

白山E功。vea¥gori白m，nei伽 Z，[A]， [BJ， norαis伽 nged.Thc singularity of [B] is a concem and can be con町OIledby thEselectionofindependent variables.ItapPE郡山atazzs responsible for decrEasein thE function while AY is rEsponsiblEfor

feasibi1ity. The above algorithm is usual1y巴xecutedfor lwo va¥ues ofα. A quadratic interpolation between the thr田 points(α= 0 is availabJe from the previous iteration)

is 日叫to detennine 出eoptimum 蹴 psize.One f伽in】施凶ali加t飽era則 onis performed for 出ioptimum stepsize.

A句01鋤 m:Genele!llzed Reduced Gradient (GR句併又旬Step 1. Choose XI (must be feasible)

N. (no. of iterations)，


p = 1 (iteration counter)

Step 2. Identify Z， Y

Ca¥culate [A]， [B]

Ca¥cuJate [GR]

Calculate Optimum Stepsizeα・ (see algorithm)

Calculate XP+l

Step 3. Convergence for GRG

If hk = 0， for k = 1， 2， . . . ， 1;

If 8j S; 0， for j = 1， 2， . . . ， m;

If KT conditions are satisfied

Then Converged， SlOp


Stopping Criteria:

LlX = Xp+1 -XP

If LlXT LlX ~ EI: Stop (design not changing)

If p = N.: Stop (maximum iterations reached)

Continue

p←p+l

Go to Step 2

Applicatlon 0' the GRG Method: The GRG algorithm is applied 10 Ex倒nple7.1. Once again the iIlustration is through a set of calculations genernted using MA1LAB. For a 5tart Example 7.1 is rewritten using a slack variable for th巴inequalityconstramt.

Minimize f(Xh x2・X3): x1-2xrX2 + xt + XIX~ -2xl + 4

Su同国tω H1(xl・X2，X3): xi+x~-2=O

H2(XhX2): O.25xi+o.75x~-1 +X3=O

O~xl~5; O~x2~5; X3~O

For山isproblem n = 2， 1 = 1， m = 1， n + m = 3，" -l = l，l + m = 2. Example 7.1 is not a good problem to iIlustrate the working of the GRG method

because出巴 onlyfeasible point where g is active is also山esolution. This makes Z， and山巴reforeGR・ascal釘.The example is retained because of uniformity. In山

implementation below， Z = [XI] and Y = [X2 X3)T.

Sec7 _3_3.m:百lIsm-fiIe handles Example 7.1 through血eGRG method symbolically. It requests a starting design which should be feasible. It sets up出eZand Y automntically and computes [A) nnd [8]. Experimenting with the code indicates that not all feasible points provide a nonsingular [8]. It then prompts the user for two v凶uωofαwhichit uses to perform a quadratic inte中olation.It computes an optimum slepsize and completes the iteration for d. Y. This ∞mpleles one iteration of白色GRG.Experimenting with白ecode iI1ustrates出atthe choices of αneed some thought for the iteration 10 succeed. The following is transcribed from the Command window for two iterations. Note出eprompts for alpha:

ltera“on 1

持 sec7_3_3Input the feasible design chosen for evaluation. [1.2 0.74833] % This is x1

The design vector [ 1.2000 0.7483 0.2200]

function and constraints(f h1 h2): 3.6304e+000 -2.2111e-006 0

A

2.4000e+000 6.0000e-001

B

7.3 DlRECT METHODS FOR CONSTRAlNED OPTIMIZATlON 301

1.4967e+000 0

1.1225e+000 1.0000e+000 s -6.0183e+000 も (ReducedGradient)

Input the stepsize for evaluation. も (Promptfor first stepsize)

0.1 alpha 1.0000e-001

No. of iterations and constraint error: 29 1.0000e-001 8.2187e-009

design vector: 5.9817e-001 1.2815e+000 -3.2110e-001

function and constraints (f h1 h2) 3.3548e+000 7.2526e-005 5.4394e-005

s '" -6.0183e+000

Input the stepsize for evaluation.

0.05 alpha 5.0000e-002

も (Promptfor second stepsize)

No. of iterations and constraint error: 12 5.0000e-002 6.2635e-009

design vector: 8.990ge・0011.0916e+000 -9.5823e-002

function and constraints (f h1 h2) 2.9702e+000 ・6.3314e-005 ・4.7485e-005

alpha

S5・6595e暢 002 も (quadraticallyinterpolatedα} =

-6.0183e+000

No. of iterations and Constraint error: 14 5.6595e-002 4.3237e-009

design vector: 8.5940e ・001 1.1231e+000 -1.3072e-001

function and constraints (f h1 h2) 2.9903e+000 ・5.2604e-005-3.9453e-005


This ∞mple出 oneiteration.This is far from satisfactory as X3， the slack variable， is negative. The stepsize and design change calculation must be mad巴sensitiveto出e

side constr冨ints.In actual implementation of the GRG， since g(X) Was inactive at出e

start， it would nol have been inc1uded in the calculations at all. Using the solution to s凶 tthe next additional iterations can be obtained leading 10 the solution.

lteration 2

Rerun Sec7 _3_3.m with X2 = [8.5940e-001 1.1231e刊)00]

The rest of出eiterations are left as an exercise for出巴 reader.To complete Sec7 _3_3.m the calculations for KT conditions must be included. In the above， since g(X) is in violation， the KT conditions were not computed. This is currently one of the more robust methods. For this pa凶cularexample the progress of solution is quile

sensitive to出estepsize chosen for interpolation.

7.3.4 Sequentlal Gradlent Restoratlon Algorlthm (SGRA)

The SGRA is a two-phase approach to出esolution of NLP problems. Starting at a feasible design， the Gradient phase decreases the value of the objective function while satisちIIng自己 linearized active constraints. This will cause cons町aintviolation for nonlinear active cons回 ints.The Restoration phase brings back the design 10 feasibility which may establish a new叩 ddifferenl set of active constraints. This cycle of two phases is悶 pealeduntil the optimum is found [15， 16]. The me出odincOJporates a d凶 centproperty with回 chcycle. Like other algorithms in血issωtion， it uses an 8Ctive ∞nstrョintstrategy. In comparis叩 theGRG combines the two phωes togethぽ.明記

method has been favorably comp81吋 toGRG and the Gradient Projection method [17]. The SGRA as originally in位。ducedin出elisted references assumes the inequality

cons町aintsof the form

gj(X)注0; j= 1，2，・…m

To be consistent wi白血ediscussion in the book it is changed to the standard form of NLP problemsぉ definedin出istext. In the following development of the algorithm， only the salient features are indicated. They are also transcribed consistent with出eother algorithms in this section. It is 隠 commended白紙 thereader review the

references for a detai¥ed description. The general problem is

Minimize

Subject to:

[X]n f(X)，

[h(X)]1 =0

[g(X)]"， ~ 0

X10wsXSXUP

(7.5)

(7.6)

(7.7)

(7.8)

同調|

調|

E F

5三

7.3 DlRECT METHODS FOR CONSTRAINED OPTIMIZATION 303

In恥 SGRAonly active in珂ualityconstraints are of interest. Equality constraints are always active. Active inequality constrainls also inc1ude violated constraints. If 11 indicates the set of active constraints，山印刷uations(7.6)加 d(7.7) are combined into

a veclor of active cons回 ints(や):

中(X)=[ぷJν (7.58)

The number of active inequality constraints is v -l. The Lagrangian for the problem can be expressed in lerms of恥 activecons回 i臨 alone[since恥 multipliersfor白e

gJ{X) < 0 will be set to zero as part of KT conditions].百leKT conditions a応 thenexpressed as

VxF(X， ν) = V x.f(X) + [V ~P'" (7.59a)

。(X)=O (7.59b)

λ吋ミO (7.59c)

where [V X+1 = [V X+I Vx中z Vx'!>v]; ν=lλ1 九世 λ.JT

GrBdient PhBse: Given a feasible design X'， find the neighboring point

主s=Xi +AX

such that 5j < 0制御=O. Imposing a quadratic constraint on the displacement dX [16]，出eproblem can be set up as an optimization subproblem whose KT conditions determine

tlX = -aV Fx(XI， A.う=αS (7.60)

The interpretation of the search direction S is used to connect to山eother algorithms in血isbook. It is not in the original development. The search direction is opposite to出egradient of the Lagrangian， which is a novel feat山宮.To compute出eLagrangian， lhe Lagrange multipliers have to be calculated. This is done by solving a system ofνIinear equations

[V ~(XIW V xj(XI) + [V ~ほう]T[V~(X')] Â." = 0 (7.61)

Steps;ze for Gradlent PhBse: The stepsizeαcalculation is based on driving出E

optimality conditions in Equation (7.59a) to zero. Therefo問， if


え=Xi-(J.Vん(Xi，i，.v)

thcn optimumα・isfound by cubic interpolation while trying ωsatisfy

VxF(え，')..l'lVxFぽ，i..I')=o (7.62)

Care must be taken白紙 thisstepsize does not cause significant constraint violation. This is巴nfol"cedby capping the squared error in the constraints by a suitable upper bound wJuch is set upω

ゆ(主g)T中(え)SPm凪(7.63)

Reference 16 suggests伽 tPm田 isrelated to ano出erperformance index Q， which is 出eerror in the optimality conditions:

Q=VFxCXi• λI')TVFxCXi， λ1') (7.64)

Restoration Phase: It is expec旬dthatat白巴巴ndof the gradient phase the function will have decreased but由巳陀 wouldbe some constraint dissatisfaction (assurning there was at least one nonlinear active constraint at the beginning of the gradient phase). The restoration phase establishes a neighboring feasible solution.1t does this by ensuring that the Iinearized constraints are feasible. Prior to血is.出eactive constraint set has to be updated (ii) since previously feasible constraints could have beC9me infeasible (and previously infeasible constraints could have become feasible) with白巴designchange caused by白egradient phase. The design vecωr and the design changes for也isphase can be written as

X，=Xg+dX， (7.65)

The design change for this phase dX， is obtained as a I伺 stsquaree.π'or in Lhe design changes subject to由esa出factionof the lin阻 rcons住aints.Setting up印 NLPsubproblem this calculation for血edesign changes is

dXr = -V xCT(Xg)σν (7.66)

Here σ"is theνvector Lagrange multiplier of the quadratic subproblem. The values for the mu1tipliers are es悩blishedthrough the linear equations

帥(支g)-VAds)TVJdE)σν=0 (7.67)

Thefactor μis a user-controlled p紅'ameterto discourage large design changes.

温〉信

書

7.:3 DIRECT METHODS FOR CONSTRAINED OPTlMIZATION 305

The Restoration phase is iteratively applied until

中(支，lCT(主，)S81 (7.68)

where EI is a small number. At the conclusion of the restoration phase白econstraints are feasible and the next

cycle of Gradient-Restoration ph筒 ec岨 beapplied again. A summary of the above steps is captured in白ealgorithm given below.

A匂orithm:Sequential Gradient Restoration A旬。'rithm(SGRA) (A7.η Step ]. Choose XJ (must be feasible)

N. (no. of iterations).

8j'S (for convergence and stopping)

p = 1 (iteration counter)

Step 2. Execute Gradient Phase

Calculate Stepsize using cubic inte中olation

Calculate xrl

Step 3. Execute Restoration Phase xP+J

Step 4. Convergence for SGRA

If hk = O. for k = 1. 2. . . . • 1; If gj S 0， for j = 1. 2. . . . • /11; If KT conditions are satisfied


Stopping Criteria:

aX = XJ*I -X!'

IfaXT企xS EI: Stop (d巴signnot changing)

If p = N.: Stop (maximum iterations reached) Continue

p←p+l

Go toStep2

Appllcation of the SGRA伺7.刀:Example 7.1 is used to test the application of the SGRA. The m・fileSec7 _3_ 4.m contains the code that applies the SGRA through a combination of symbolic and numeric calculations. As in山eprevious ex創nplesit is customized for two variables. one equa1ity and one inequality constraint. Unlike the GRG implementation it incorporates白eactive constraint strategy. It executes one cycle of the SGRA which has a single iteration in自己 Gradientphase and several iterations in山eRestoration phase. In the gradient phase the cubic interpolation of the original technique is陀 placedby the quadratic interpolation used everywhere in the book. The starting input is a feasible design vector. In出egradient phase the method


reques包 avalue for the stepsize to scan for the optimum ぱ usingquadratic int巴中olation.This is important for successive cycles start at the optimum value of the stepsize for the previous cycl巴.The value ofμin Equation (7.67) is set at 0.5. Printing is controlIed出roughthe semicolon and白ereader is encouragedωfollow details by f宮町lovmg11.

In the folIowing the program is run wi白山estarting design used for the GRG method.百1巴 outputfrom the Command window is pasted below with some bold annolations 10 draw auention.

>> Sec7 3 4 工npuヒヒhefeasible design chosen for evaluation. [1. 2 0.74833] (feasible design vector input)

The design vector [ 1.2000 0.7483]

function and constraints(f h g): 3.6304e+000 -2.2111e-006 -2.2000e-001

*********************** Gradient Phase

*******************合合**

The Lagrange multipliers -1.0812e+000

Input the stepsize for evaluation. 1 (alpha used to start scanning process)

alpha for quadratic interpolation o 1.2500e-001 2.5000e・001

(alpha that brackets the minimum) function values for quadratic interpolation 1.0142e+001 6.6291e+000 1.1300e+001

(corresponding function value) optimum alpha: 1.1615e-001


function and constraints(f h1 h2) : 3.0077e+000司 1.3681e-001 9.8328e-002

(both constraints violated) The performance indices Q， P 6.3181e+000 1.8718e-002

(P measures the squared error in the constraints) 合**********************

Restoration Phase ************会合*********

Number of Restoration iterations: 21


/蓮|

ヨ!

世 l

包

是


function and constraints(f h1 h2): 3.0000e+000 6.712ge-005 4.6741e-005

(Constraints satisfied -9 active) The performance index P 6.6911e-009

The solution has converged凶 oneiteration. The SGRA is of the same caliber as th巴GRG (some comparisons show it to be better). It is difficult to compare different algorithms because of differenl coding struclures and nonstandard implementation details. Nevertheless the SGRA has considerable merit. It is definitely better than the GRG as far as Example 7.1 is concemed (Section 7.3.3). This comparison may not be fair s~nce the active constraint strategy was not used for the GRG even though it is part of the GRG. This demonstrates the need for such a strategy in all algorithms白atdealwi白 cons回 inedoptimization.


Three additional examples are introduced here 10 ilIustrate and expand the凶 eof由巳setechniques. During the early development of these techniquω由巳rewas a significant emphasis on efficiency， robustness， and keeping computational resources as low as possible. Most of山etechniques were developed on mainframes using FORTRAN. Today Ihe search for global solulions (出atare largely heuristic)， coupled with simple calculations rep伺 ledalmost endlessly， performed on dωktop PCs wi白 incrediblecomputing power， Ihrough softwru芭 systems出alprovide a wide range of resource唱has shifted the focus to experimentation and creativity. The author feels such an effort must be coupled with knowledge and insight gained through a wide selection of applications佃 dunderstanding白eprogress of numerical calculation. In spite of all 白eexpos町 eso far in the民民 formany real design problems the solution is not automatic. D国 lingwith nonlinear equations is a challenge and often surprising. Trying 10 understand why the lechnique does not work is as much a learning expel"色nceωaroutine solution to the problem. Since lhese examples use the melhods of山ep問vioussection with differ芭ntfunctions， all of the code developed before is revised with and the appropriate changes and renamed.泊施semodifications偲quirethat few lines be changed from the original code.百lisallows the user to make incremental changes as needed. This is not an efficient approach bUl capitalizes on the unlimited hard drive space available today. Students are urged to understand the algorithm， the flow of the code， and出eresult being sought.

7.4.1 Example 7.2一円agpoleProblem

This is the same problem which was solved graphicalIy in Chapter 2， analytically in Chapter 4， and now considered for numerical solution. In出issubseclion the scaled version of Ihe problem is solved. From Chapter 4， Example 4.3， it is reintroduced as

308 NUMERICAL TECHNIQUES FOR CONSTRAINED OPTlMIZA TION

Example久2

Minimize f= xT-~ (7.69)

gl: 2.55ωヰ+O.l36lx1 -33.148(x1-x~) ~ 0 (7.70吋

g2: (0.似 88+ 1.4619x1 )(イ +x内+~ー 166.5614(x~ -x~) ~ 0 (7.70b)

む1.0868x1+ 0.3482 -28細川-~) ~O (7.7Oc)

g4: x2-x1 +0.001 ~O

0.02 ~ x 1 ~ 1.0; 0.02 ~ x2 ~ 1.0

(7.70d)

(7.71)

Ins勿htabout Numerical Solution: In Chapter 4 it was investigated出創出esoIution does not satisfy the KT conditions because th巴solutionis not a regular point. It was ascertained伽 t曲egradients of出etwo active constraints伽 tdetermine白esolution (810 g3) are nearly parallel in the neighborhood ofthe solution. How will this affect the numerical search for the soIution?

TypicaJly what a numerical technique does is pick a useful direction and see how best it can move along it. Suppose the technique decides to move along the gradient of 83 to develop it into an active constraint. Thcre is obviously going to be a problem in pushing gl to become active. Pushing a constraint to be active is a basic way to reduce the objective function in well-formulated NLP problem. Therefore， most algorithms (unless曲eyare trained to compromise in some significant way) wilI underachieve by only pushing one ofthe constraints to be active and ensuring the other is feasible.

ALM_7 _ 4_1.m: This m-fiIe uses the ALM method 10 solve山eproblem. ln order to preserve the old ALM.m to 陀 flectthe discussion in Section 7.2.2 a copy of the file is made. Ofun341.m， Hfun341.m， Gfun341.m， and FALM341.m reflect白efunctions from Example 7.2. A1so， ALM_7_4_1.m is adjusted ωavoid handling equality cons町aints叩 dprovide some printing support. 0山erwiseit is essentially the same. Once again the output from the Command window is p邸 tedbelow and annotated to draw attention to some issues. Only由巴 initialprompts wi山theinputs， the initial iteration， and the consolidated information after the final itemtion眠 copiedbelow.

持 ALM_7_4_1 (invoke program) Input the starting design vector This is mandatory as there is no default vector setup

:~.Ui

7.4 AOOlnONAL EXAMPLES 309

The length of your vector indicates the number of unknowns (n) Please enter it now and hit return for example [ 1 2 3 ...]

[0.75 0.55] (initial design vector)

The initial design vector: 7.5000e-001 5.5000e-001

Input the minimum values for the design vector. These are input as a vector. The defa~lt values are 3 * start value below the start value unless it is zero. In that case it is -5: [0.02 0.02] (input lower bound not currently used)

The minimum values for the design variables are 2.0000e-002 2.0000e-002

Input the maximum values for the design vector. These are input as a vector. The defau1t va1ues are 3 * start value above the start value. 工fstart value is 0 then it is +5: [1 1] (input upper bound -currently not used)

The maximum values for the design variables are 1 1

Number of equality constraints [0] (default value of 0 is ok)

Number of inequality constraints [0] 4 Initial values for beta [1 1 1 1] (input number of inequality constraints and initial beta)

ALM iteration number: 0 Design Vector (X) 7.5000e-001 5.5000e・001Objective function 2.6000e-001 (In the following violated constraints are shown in bold) Inequality constraints -5.9180e+000 -3.5997e+001 5.2383e+000 -1.9900e-001 Square Error in constraints(h， g) 0 0 Lagrange Multipliers (lambda beta): 0 1 1 1 1 Penalty Multipliers (rh rg): 1 1


ALM iteration number: 4 Design Vector (X) 6.4726e・0016.1013e-OOl Objective function: 4.6695e-002 Error in constraints (h， g) 0 2.1366e・009Lagrange Multipliers (lambda beta): 0 0 4.8227e・002o Penalty Multipliers (rh rg): 1000 1000

The values for x and f and 9 are 7.5000e-001 5.5000e-001 2.6000e-001 0 6.5647e-00l 6.0173e-001 6.8871e-002 0 6.4726e-001 6.1017e-001 4.6638e-002 1.5525e-006 6.4726e-001 6.1013e・0014.6695e-002 2.1366e-009 6.4726e-001 6.1013e-001 4.6695e-002 2.1366e・009(Note design and function are not changing hence

stopped) The values for beta are 1.0000e+000 0 0 0 1.0000e+000 0 0 0 1.0000e+000 1.4062e-002 3.B9B3e-002 4.8227e-002 1.0000e+000 8.9252e・0011.7073e-001 0 (Each column represents values of beta for an

iteration -beta for constraints 1，2 and 4 are zero and it can be seen below they are feasib1e but inactive. Only the third constraint is active)

Inequality constraints ・6.7858e-002 -4.9734e+OOO 4.6223e-005 -3.6136e-002 (Note third constraint is active. The first one is

almost activel

The analytical solution for出isproblem is

X;=O后774; 弓=0.6445; I = 4.3577E -02;

g;-O.O; g;=-4.9837; g;=O.O; g;=-O.319也 -02

In comparison， the ALM has done a decen1job of geuing cJose 10 the solution befo悶staIling. Some of白iscan be traced 10自己 discussionprior 10 running山eprogram.

7.4.2 Example 7.与一BeamDeslgn

This sec1ion r叫uires白euse of出eOptimization Toolbox from MATLAB. More specificaJly， it uses 1he QP function ωsolve 1he QP subproblem in the SQP me曲。d.The problem is solved using the direct proced町 eilIustraled through the example in

調|7.4 ADDITIONAl EXAMPLES 311

Section 7.3.2.噴出sectioncombines the symbolic capability ofMA札油明白theresou民凶

available in the Optimization Toolbox to implement an origina1 interpretation of the SQP. 泊施example is白edesign of an I-beam for use in a par討c叫ars回 ctura1problem.百出isa回 ditiona1m∞hanicaVcivil engineering strucωra1 optimization problem. It was introduced in Chapter 1 for ilIus回 tion.It a1so appears in Chapters 8佃 dIO for iIlus回 tionof additiona1 issues. In血eexample here，出eI-b担 mwiI1回rrya weight at福田R包r.百le田 n凶cload泊g佃 d血edesi伊 variablesfor出eproblem are defu:ted加Figure8.7 (Chap町8).百lefigure is not reproduced he時 butX I is the depth of the beam， X2 is the flange width， X3 isthew出血ickn郎.四dX4is出ef1ange tJucknωs. Other parameters (g，ムW.E，σpτy)arenotr吋 efinedhe毘.

百lemathematica1 model is

Minirnize J(X) = pLAc

Subjec1 to: gl(X): WL3 - 12EI ~ 0.25 (de自国tionconstraint)

82(X): WLxI -8/1σy ~ 0 (normaI stress constraint)

g3(X): WOc -21x3τ:y:S; 0 (sh間町ssco田町血t)

Some geometric cons1raints to relate the various design variables are

g4(X): XI -3X2 ~ 0

8'(X): 2x2 -XI ~ 0

g6(X): X3 - 1.5x4 ~ 0

g7(X): 0.5x4 -X3 ~ 0

The side constraints on出edesign variables are

3 :S;x1 ~ 20， 2 ~X2 :S; 15， 0.125 :S;x3 ~ 0.75， 0.25 :S;x4:S; 1.25

The following relations define出ecross-seclIonal area (Ac)， moment of iner由(乃，印d伝説momentof area abou1 the centroid (Qc) in terms of the design variables:

Ac=2らX4+x1x3 -2x3x4

Iーらxf (ろ-x~(xl -2x3)3 一一ー

12 12

Qc=0.5x2ぬいI-xJ + 0.5x3(x 1 -xJ2

The design parameters for the problem are W = 1.4 x 2000 (1.4 is山巴factorof safety)， L = 96， E = 29 E+06， p = 0.284，σy= 36 E+03，τ:，=21 E+03.


EX343.m: This is出em・file由alwill implement the Solulion lhrough the SQP.

While this is a traditionaI design problem. it is aIso nol trivial. It is rare to find a

published detailed procedu問 forsolution. The solution is usuaIly obtained

numerically. In any event. anaIyticaI application of the KT conditions is usuaIly not .

contemplated because of 27 cぉω(for山e7 constraints) even if many of those cases

are trivial. IncIusion of the side constraints will provide an additionaI 8 linear

cons剛山(agrandto凶 of15 constraints) of the form (iIIus凶 tedfor X) onlyぉ)

gs(x): -x1 + 3 S 0

gI2(X): xl-20S0

The reader is strongly recommended 10 review出巴codein the m-fiIe for由efollowingreasons:

・ The power of symbolic manipulation is回 ploitedfurther using vector definitions 10 make the implementation fairly easy. The symbolic calculations are used to genemte山edala for the QP subproblem.

• Th巴 symbolic calculations are effortlessly inlegraled with numerical

computations (th巴soluLionto the QP is numerical).

• The implementation of恥 SQPis s凶 ightforwardand direct (and simple).

・τ'hesolution is si伊ificantlybetter白anthe numerical solution by MATL油、own SQP program (Chap町 10).

The program requires th巴startingguess for the design vector. The functions for the

example ar芭 hardcoded. The final solution is printed to the Command window:

Initial Guess: Final Design: Objective:

nunu

∞お'anu

nuεJ

nU司

4

zJ'i

nunU

側

側

εJ弓4

nu『

roo

nuzutJ

nu

，U06

m

5判

AlI constraints ar芭 satisfied:

Ac¥ive constraints: 1 (deflection). 7.9 (X2ー lowerbound).10(.均一lowerbo叩 d).

11 (X4ー lowerbound)

The reader is encouraged to explore why the solution obtained he問 issignificant1y

better出回 theone using the SQP from the toolbox. granted that the implementations

are not the same. Some thought should be given to山 penormanceof quasi-Newton

(me¥ric) methods and their adaptation ¥0 constrained lechniqu巴sif the Hessian is no¥

positive definite， which in this case it is.


7.4.3 Example 7.4-0ptlmal Control

This section introduces another cIass of problems白紙 C加 beaddressed by design optirnization techniques. It is the basic trajec¥ory determination problem in optimaI control. Given two points in a plane， find a pa出血atwi1I be tr温versedbetween them using a steering v田 torof constant magnitude. The path has to be釘aversedin unit

time. The probl巴mc叩 bedescribed using Figure 7.11. If the problem回 nbe described in出ephysical plane. the sl田 ringvector considered as velocity.出enthe kinematic

叫 uationsfor a particIe moving on白epa山 canbe described by the following differential equations (d邸 i伊 optimizationprefers to deal with a1gebraic functions). whe隠 X.Ylocate出eproblem in physical space:

土=V cos 8(t); 0 S t S 1

タ=V sin 8(1); 0 S t S 1

The initiaI and final conditions are (t11ese were the values chosen for the example)

λ(0) = 0; y(O) = 0

x(l) = 3; y(l) = 2

Here. 8(t) is regard巴das a control variable. II is a continuous function of t between 0 S t S 1 that will transfer the pa凶c1efrom自己 initialpoint 10 the final point. In contrast. design optimization (i.e.. ¥his book) determines scalar values of the design variables.

These types of problems belong to the area of optimal control. Th巴閃 arespecial

numerical techniques出alare available 10 solve Ihese categories of problems. One

approach to these problems is to use design optirnization techniques. This is aided by using a parame町icrepresentation for the control function. Many different par滋ne凶c

隠p問 sentationsfor the control have been used. such as cubic splines and Hermite polynornials. In this s¥udy a Bezier curvc is used to rep陀sentthe control. The differential equations ar宮山ennumerically inlegrated 10 es阻blishth巴町司jeclory.

y

x

FIgure7川 Descriptlonof Irajeclory∞nlrol.

1告書.

書軍

which requires brachi.m to retum the state space definition of由巳 problem.

brachi.m also includes the Bezier interpretation code. Figure 7.12 describes出eBezier ve氏icesof the control function at the stan and出巴

final iterations. The final continuous control is also plotted on the figure. The start curve is a s位aightline through the vertices and is not shown. Figure 7.13 shows白estart trajectory組 d出efinal位司jectory.It is cl回 rfrom白efigure白紙白einitial and曲efinal conditions are satisfied for the final tr司jectory.Since白eequations we陀integrated. the differential constraInts are therefore completely satisfied.

The final tr吋ectoryshown is only feasibleー itis one of many一白紙 canp蹴

betw田 nthe two points. This opens the possibility山ata further op自国国tionc卸 beperformed with毘 spectto出巴佐高jectory.Forexample. find出eshortest path in出etimeprovided. Not巴 itis not白estraight pa山 sincefor a given velocity magnitude the destination may be犯 achedsooner山anp問 scribed.The c¥assical problem in optimal ∞ntrol is曲ebrachislochrone problem. lt is出eminimum time for the p紅白le10 tr祖velbetween two prescribed points dynamically (obeying principles of dynamics). The reader is cncouraged to expand由巳 techniqueand山ecode to explore自己seadditiona¥ interesting problems.

315 7.4 ADDITIONAl EXAMPLES NUMERICAL TECHNIQUES FOR CONSTRAINED OPTIMIZATION

Solutlon Technique (adopted in出issection)

• Use a Bezier curve to parametrize the control function.

• Integrate the differential equations forward using a Runge-Kutta method (ode45.m)

• Use叩 uncons回 inedlechnique (fminunc.m) minimizer to命ive山esquare of 出eerrorin白efinal condi曲目 tozero 0回 stsquar吋 error).

314

Feaslble troJectory

This example nωds the MATI.AO toolbox to execute fminunc.m. Altemately.出eunconstrained minimizer can be among those previously developed.百 istranslates inlo an uncons回 inedminimum problem， which therefore should have been handled in Chapter 6. It is inc¥uded here because山edifferential equations are actually considered d留量rentialconslraints. The objective function therefore is

f{X}= [ズ'finnl-x(l)f+ [Yfinal-y(l)J2

Ex744.m will execute the example and produce plots of the controI and the tr司j田 tory.It uses two standard MATLAO functions mentioned before (odc45.m and fminunc.m). The Bezier curve generation r叫 uirescoeff.m. Comblnatlon.m. Factorlal.m.百leobjective function is constructed in obLoptcont.m. It陀 qui問sthe use of ode45.m.

M泊泊語ze

The， vertlces and 0 (1)

z ，，，，， llnal polnl ，

，

3

3.5 2

1.5

2.5

，

，

，

，

，

，

，

，

，

，

，

，

，

，

，

，

，

，

，

，

，

，

，

，

，

，

'

， -

H

---圃

-

2

>-1.5

0.5

3.5 3 2.5

Flgu陪 7.13 Trajeclories→tart則前nal.

2 x

1.5 0.5 -0.5

0

。

o

。

o

0.5

，

，

，

，

，

，

，

，

，

，

，，

，

，，

，，

，

，

a'

，e d

y

o

，，

，

，

a' ，

，

，

，

，

，

，

，

，

，

，

，

，

，

，

，

，

ー-...

0.9 O.自0.7 0.6

Control funclion 0(/)，

0.5

tlme

0.4

Flgu悶 7.12

0.3 0.2 0.1


REFERENCES

1. Fiacco， A. V.， and McConnick， G. P.， Nonlinear ProgralllmiJlg・Sequential

Unconstrailled Minimizotioll Techniques， Wiley， New York， 1968.

2. V畑氏rpl細胞， G. N.， Numerical Optimizotion Techlliques for Engineering Desigll， McGraw-HiII New York， 1984.

3. ANSYS Software， Ansys Inc.， Pittsburgh， PA.

4. Bertekas， D. P.， Constrained Optimizotion and Lagrange Methods， Academic Press， New York，1982.

5. KeJley， J. E.， The Cutting Plan巴Me白od，JoumalofSIAM， Vol. 8， pp. 702-712，1960.

6. Gill， P. E.， Murray. W.， and Wright， M. H.， Practical Optimizotion， Academic Press， New York，1981.

7. Boo1， J. C. G.， Quadrauc Programming， Sludies itl Mathematical and MQllagerial

Economics， H. Theil (巴dよVol.2， North-Holland， Amsterd町n，1964.

8. Biggs， M. C.， Constr泊nedMinimization Using Recursive Quadrauc Programming: Some Altemate Subproblem Fonnulauons， Towards Global Opt初IIzotion，L. C. W. Dixon and G. P. Szego (eds.)， pp. 341-349， North-Holland， Amsterdwn， 1975.

9. Powell， M. J. D.， A Fast Algorithm for Nonlinear Conslrained Optimizauon CaIculalions， No. DAMPrP77INA 2， University of Cambridge， England， 1977.

10. Han， S. P.， A G10bally Convergent Method for Nonlinear Programming， JoumalザOptimizotion Tlreory and Applications， Vol. 22， p. 297， 1977.

11. Arora. J. S.. Intrnductiolllo Oplimwn De.risn， McGrnw-HilI， New York， 1989.

12. Branch， M. A.， and Grace， A.， Optimizotiotl Tooめ'ox，User's Guide， MathWorks lnc.， 1996.

13. Wolfe， P.， M白血odsof Nonlinear Programming， Recenl AdvQllces i/l Mathemalical Programming， R. L. Graves and P. Wolfe (eds.)， McGrnw-HiIl， New York， 1963.

14. Gubriele， G. A.， Dnd RagsdelJ， K. M.，百leGcnerolized Gradlcnt Method: A Reliable Tool for Op伽aIDesign. ASME Joumal of Engineering and IndJ凶り"SeT加 B，Vol. 99， May 19η.

15. Miele，A吋 Huang.H.Y.，加dHeideman，J.C.，S叫uentiaIGradient Restorauon AIgorithm for the Minimizauon of Constrained FunctionsーOrdinaryand Conjugate Gradient Versions， Joumal ofOptimizotion Theoηand Applications・， Vol. 4， No. 4，1969.

16. Levy， A. V.， and Gomez S.， SequentiaI Gradient-Restoration AIgorithm for the Optimizauon of a Nonlinωr Constrained Function. Joumalψ'he Astronautical Scietlces， Speciallssue on Numωic叫Me曲。dsin Optimizauon， Dedicated to Angelo Miele， K. H. Well (sp. ed.)， Vol. XXX， No. 2. 1982.

17. Rosen， J. B.， The Gradient Projection Method for Nonlinear Programming， Part D: Nonlinear Constraints， SIAM Joumal of Applied Mathematics， Vol. 9， No. 4， 1961.

PROBLEMS

7.1 Modify Sec7_2_1.J)loLm to review contour values and plot automatically selccted contours.

7.2 Modify Sec7 _2_1_calc.m to automatically calculate the initial multipliers， loop over pena1ty multipliers. and to automatically pick the design vector白紙

satisfies the side constraints.

PROBLEMS 317

7~ (uses Optimization Toolbox) Develop山eSLP by using the LP solver合omthe

Optimization Toolbox and apply to Example 7.1.

7.4 Solve Exampl巴7.1by solving山eQP problem graphica1ly.

7.5 Dcvelop自己 codeto find a feasible solution from a given slarting vector for

叩 nslt司inedproblems.

7.6 Develop a program to calculate the multipliers for the NLP problem at a given

design.

7.7 Develop a program for a constrained stepsize calculation procedure.

7.8 (uses Optimization Toolbox) Use the QP program from the toolbox and develop

your own SQP implementation.

7.9 (uses Optimization Toolbox) Use出eSQP program from MATLAB to solve

Example 7.1.

7.10 Develop Sec7 _3_2.m to automatica11y execute severa1 iterations to

convergence.

7.11 Implement a new version of SQP where the Hessian matrix is maintained as the

identity matrix for a11 iterations.

7.12 Finish solving Example 7.1 using the GRG me出od.

7.13 Solve Example 7.1 with Z = [X2] and Y = [x， x)]T. 7.14 Modify the GRG code to include consideration of active constraints and

comp脱出eperform剖 ce.

7.15 Build in KT condition calculation into the SGRA.

7.16 Solve the Brachistochrone optimal control problem.

8

DISCRETE OPTIMIZATION

This chapter imroduces some concepts， methods， and algorithms associated wi白discrete opti凶 zation.It is not possible to compress出巴 su均配tof Discrete optimizauon to a single chapter as the topic itself is capable of spawning several

courses. Discrete optimization is very different， difficult， diverse， and continues to develop even today. These problems are largely combinatorial and are

compu凶 ona1lymore time intensive白叩恥 correspondingcontinuous problems. The problems addressed by the discrete optimiz凶 onresearch community are mainly in 白巴釘e伺aofop戸eratioロ回s児邸s鈍e釘卸制chch紅回a低c低削riz吋 usually by li泊nea訂r口mo

optimizatio叩nmostlyinc∞or中porat，匂.esno叩nline伺ar口re芭叫la瓜副tio叩n】IS.From a real perspecti ve， discrete design vaItablesMBfundamental inengineedng optimization.h出ebeam desIlm

p帥 l巴m in Chap総r1， a p削 ical solution should involv巴 identifying an ‘off-the-shelf' beam as出erolling rnill will probably make it prohibitively expensive for a limited production of出巴 uniquebeam白紙wasidentified路 thesolution to出e

continuous optimization problem. In the problem regarding白enumber of different placement machines the solution was expected to be integers (discrete value).

平rnilarl円choiceof dian附 rs，lengths， washers， valves， men， components，陶ckSlzeS，叩Gso on，are usually guided by avoiding high procurement costs associated

wi曲 nons凶 dardcomponents unless cost recovery is possible because of large volumes. Co叩n凶】u刷ti加IOl附Il

item路sto be completely manufactured i加nル帥"占h加ou凶s巴 (nooflι恥恥lfcom戸 nenlis necessary)・

Practical engine泡ringdesign陀quirl邸由alsome design variables belong (0 an

ordered set of values.-discrete variables.This makes it a discrete optimization 318

DISCRETE OPTIMIZATION 319

problem. However， there are very few engineering problems白紙 arecurrently

solved as a discrete optimization problem. First， discrete optimization algorithms are difficull to apply. Second， they are time consuming. Third， most discrete algorithms and code address linear mathematical models. Discrete optimization in engineering wil1 necessarily involve adaptations from白ecurrently available techniques used by the decision-making communily. Such adaptations ar巴uncommon. It is also rare to find叩 ybook on design optimization白紙 addressesor includes the subject of nonlinear discrete optimization， notwithstanding the fact 出atit is enormous in extent. A modest effort is being made in this book to acquaint the reader with the subj巴ctarea.

百letypica1 approach for incorporating discrete design variables in engineering is to solve the corresponding continuous optimization problem and adjust the optimal design to出巴 nearestdiscrete values (白isis sirnilar to the rounding process to arrive at an integer number). According to Fletcher [1]， there is no guarante巴白紙白isprocess

is correct or白紙 agood solution can be obtained出isway. Very often出isrounding

may result in infeasible designs. Given白紙 thealtemative is to solve the discrete optirnization problem itself， a systematic approach to山isrounding process has become acceptable. A rounding process白紙isbased on maintaining a feasible design is available in Reference 2. τ'his chapter and the next will differ significantly合omthe previous chapters in both

白血 contentand organization. Their primary focus is presenting new ideas and content rather白andeveloping a technique or assisting in code dev巴lopment.Discrete problems require significantly different concepts山間出etraditional continuous mathematical models of the earlier chapters. Simple examples are used to bring out 白edifference. For example， derivatives， gradients， and Hessian do not apply to discrete problems. By extension， search directions and one-dimensional stepsize computation lose their relevance. While白ereare s巴veralexcel1ent references on discrete optimization， almost all of them deal with linear mathematical models. There are few， if any， about nonlinear discrete optimization applied to engineering problems.

Discrete optirnization problems (DP) are implied when the design variables are not continuous. Extending出巴conceptof the side constraints， these design variabl

加tegervalue of 8 => binary value of 1000

320 DISCRETE OPTIMIZATION

Therefore， if出edesign variable x is res凶ctedto integers between 0 S; x S; 16， then x

can be問 placedby five 0-1 design variables [y.. Y2， Y3， Y4・YsJfrom which x c加 b巴assembled as

x = YI (24) + Y2 (23) + Y3 (22) + Y4 (21) + YS (20)

Such a transfoIIDation is not recommended if白巴reis a large number of integer variables. Engineering design problems can be expected to contain both continuous and discre飽 variables.These are termed Mixed Programming (MP) problems if出emathematical model is line低 Inthis book the classi白cationis extended to nonlinear problems too.

In the interest of manageability， only three methods釘 epresented in出ischapter. These methods are only representative and in no way address由巳 topicof discrete op山nizationsuf白ciently，let alone completely. The third one has seen limited use， but with compu匂tionalresources not b巴inga hurdle today， it is definitely attrac首ve.The selection does not represent any priority， though血emethods are among those出athave been applied often in engineering. The methods of the next chapter have evolved through their initial application to discrete optimization. They could also belong to 血ischapter but have been kept sep紅 atebecause出eyare the driving force in the search for global optimum today. The methods of出ischapter are (l) E油 austiveEnumeration， (2) Branch and Bound (partial or selective enumeration)， and (3) D戸IamicProgramming.

8.1 CONCEPTS IN DISCRETE PROGRAMMING

In the opening discussion some defmitions of disc閃teprogramming problems， regarding the nature of discrete optimization were in紅oduced.ln出issection the concepts are detailed using a simple unconstrained minimization example. As a result，白eoptimality issu回 donot in加 deon presenting the important ideas in discrete optimization including the treatment of finding a continuous solution followed by adjusting the discrete design variables to neighboring discrete values.

Example 8.1 Minimize出巴objectivefunction given below， wherexl is a continuous variable and X2， X3 are discrete variables. X2 must have a value from the set [0.5 1.5 2.5 3.5J and x3 must have a value合umthe set [0.22 0.75 1.732.24 2.78J.

Minimizef(x1・1ヤx3)= (x1 -2)2 + (x1 -~)2 + (x1 -X3)2 + (x2 -x3)2 (8.1)

The side constraints on the design variables can be set up as

x1 E R (8.2a)

X2 E卯.5 1.5 2.5 3.5J E xu (8.2b)

自.1 CONCEPTS IN OISCRETE PROGRAMMING 321

x3ε[0.22 0.75 1.73 2.24 2.78] E X3d (8.2c)

The symbol E identifies出atthe variable on the left can have one of the values on the right side. In Equation (8.2a)， R stands for a real value. This represents a standard use of自己symbolE加 dis an effective way of expressing the idea that the discrete values C佃 onlybe selected from a group of values. Tbis is a mixed programming (MP)

problem.

8.1.1 Problem Relaxatlon

日吉霊抗日三宮司直P44MJ均三主

E出iswere a unconstrained problem in continuous variables， the solution could be obtained by setting V f = 0 and sol ving白e由民eequations for白evalu巴sof x.. X2' and X3・Eq回tions8.2b and 8.2c h鎚 tobe overlooked. Altemately， for a continuous problem， inspection of the objective function yields白efollowing solution:

x;=2; x;=2; x;=2; I=O (8.3)

For the original problem伽'dXlis defined， but not df/dx2 or df/dx3' since X2佃 dX3are discrete values. Derivatives are defmed by taking the limit of the ratio of change of objective functionfto the change in the value of the design variable， as the change in the variable approaches zero. The value of由巳objectivefunctionfis only defined at selected combination of X2・X3in Equation (8.2) and is noL defined elsewhere. Smalllinfinitesimal changes in the discrete design variables and therefore in由eobjective function are not defined in Example 8.1. Derivatives with respect to恥

discrete variables do not eXIst. This conclusion is of m句01'significance as it makes this book's previous body of work of Iimited use in出epursuit of tbe solution to the DP. To recollect，白enecess紅yand sufficient conditions， which drove the algorithms， were based on the gradients and their derivatives. Two areas in the previous chapters escape with a limited impact of曲isstatement.τbeyar芭 LPand zero-order methods for numerical solution to unconstrained nonlinear optimization. Both of them play a significant role in discrete optimization problems. It is therefore possible to conclude 出atsolution to Example 8.1ぉ establishedin Equation (8.3) is incorrect (出eside constraints are in violation).

In DP the solution in Equation (8.3) represents the solution to a relaxation problem. Problem relaxation can take severalforms. Mostly it is applied to由erelax副onor白eweakening of the cons町ain匂 orthe objective functions. There眠 noexplicit constraints in Example 8.1. In出isinstance， the relaxation refers to the removal of the restriction of discreleness ofthe variables. This is identified as continuous relaxation. The rel出 edproblem or relaxation has several advantages [3].

. If a constraint rl巴laxationis infeasible， so is the problemlmodel it relaxes.

・Constraintrelaxation expands the set of feasible solutions. The relaxed optimal value must improve or equal the best feasible solution 10出eoriginal problemlmodel.


• The optimal value of any relaxed model provides a lower bound on山eoptimal

solution if il is a minimization problem. Similarly il establishes an upper bound

for maximization problems.

• Many relaxation models produc巴optimalsolutions山atare easily rounded to

good feasible solutions of the original problemlmodel.羽lisappears to創刊

discrete optimiz副onfor engineering design problems.

The values established in Equation (8.3) are a solution to由巳continuolIsrelaxation of Example 8.1. It was easy to obtain出issolution while a technique for the solution to 白ediscrete un∞出国inedop也凶zation is still unknown一白eKuhn-Tucker conditions are no longer useful.百lesolution in Equation (8.3) is not acceptable-xi and xj are not elements of the permissible se1 (8.2). The value of I = 0 will be betler 白剖恥bestdiscrete solution-lower bound 0" the solution 10批 originalproblem.

8.1.2 Dlscrete C!ptlmal Solutlon

A standard approach 10 solving discre1e optimization problem， particularly in engineering， is 10 use the continuous relaxation of the mathematical model and genemte a continuous solution. For Example 8.1 this is given in Equation (8.3). More than likely出ediscrete variables will not belong to the p悶defineddiscrete sets. These variables are then changed (or rounded) to山enearest discrete values. For problems

with constraints， feasibility is checked. The best feasible solution is lhen chosen. Unlike continuous problems， there a問 nonecessary and sufficient conditions to

satisfy. ln白ecase of Example 8.1， Equations (8.2) and (8.3) provide the relevant

information to establish the solution. For convenience， tJley are reproduced here:

XJ εR (8.2a)

X2 e [0.5 1.5 2.5 3.5] e X2d (8.2b)

X3ε[0.22 0.75 1.73 2.24 2.78] E X3d (8.2c)

(8.3) x; = 2; X; = 2; x; = 2; I = 0

ηle continuous variable XJ is chosen 10 have a value of 2.0. Figu児 8.1indicates four sets of disc附 evalues for X2 and X3 around the continuous solution. The least value of

the objective function at these points will be considered白esolution. Evaluating the

values of the objective釦nctionat those points:

x; = 2; X; = 1.5; X; = 1.73; 1=0.3758 (8.4a)

x; = 2; x; = 1.5; X; = 2.24; 1=0.8552 (8.4b)

掴 l'

8.1 CONCEPTS IN DISCRETE PROGRAMMING 323

3

'1+'111ιt'

-e'iri-

-''白'17iboa

--tJo--T

，A，t・・1

「

ド

llφ1lrー

内

4

局制

帽

，

-

-

-

同同曲

-a圏一』園、，由一言w』U曲

-o

0.5

ーーーー守一一ーーーーーーー叶ーーーーーーー吋ー一一ー一一一一一一

ーーーー」ーーーーーーーーー _L__ーー十ーーーー」ーーーーーーーーー・戸川

。。五

0.5 1.5 2 2.5 Dlscrete vorloble ~

Flgure 8.1 Dlscrete nelghborhood 01 continuous solution.

3.5

X~ =2; X; = 2ふ xi= 1.73; ] = 0.9158 (8.4c)

x~ = 2; x; = 2.5; xi = 2.24; ] = 0.3752 (8.4d)

From this exercise，出e同 8tvalue of山eobjective function is available in Equation (8.4d) and would be regarded as the adjusted optimum solution to the original MP problem.百lisis also the common practice in engineering design optimization.

The best solution， however， is obtained at

x;= 1.7433; X;= 1.5; x;= 1.73; ]=0.1782 (8.5)

The difference between Equations (8.5) and (8.4a)， only in白evariable XI， suggests 伽 tadditional continuous optimization needs to be performed for each set of曲E

discrete values selected， confirming the回 rlierobservations by Fletcher.百usoptimization should be easier as the order of the mathematical model wi¥l be 問duced

by the number of discrete variables since they have beenωsigned numerical values.

This simple example問 commendsat血every least a three-step procedwちforproblems

白紙inc¥udediscrete variables:

Step 1: A conlInuous relaxation that identifies several sets of discrete variables for

further explor副 on.

324 OISCRETE OPTIMIZATION

Step 2: For each such set of discrete variable combination， a continuous

optimization is performed 10 establish a new optimum va1ue of the continuous variables and the corresponding objective function. If all of lhe variables are discrete， then only the function and constraInts need to be eva1uated at each of the sel of variables.

Step 3: A simple comp釘isonof the above solutions/va1ues in Step 2 to identify出e

optimum solution to山ediscrete problem.

This unconstrained optimization example in three variables has demonstrated

that discrete optimization requires a lot of work compared to continuous

optimization. This work expands significantly if the number of variables increases

or if the mathematical model is enhanced through the inclusion of constraints.

Another essential fl回旬rein the above exploration is白紙 nonew mathematica1 conditions were necessary for巴stablishmentof the discrete solution beyond a

simple comparison of the objeclive function. The nature of discrete variables and

discrele functions precludcs any sophisticatcd mathematical conditions

established by derivatives of the functions involved in tbe model. Trapping and

branching based on comparison of values are the mainstay of discrete algorithms.

There techniques are classified as heuristic methods. This encourages unique and personal implementations of the search for discrete optimization that can h巴

tailored for a class of problems.

8.2 DISCRETE OPTlMIZATION TECHNIQUES

There are three discrete optimization techniques in this section. The first one is

Exhaustive Enumeration. This involves identifying tbe solution to白e

mathematical model for all possible combinations of the discrete variables. This

is suggestive of the zero-order numerical techniques of Chapter 6. Those

methods lackcd sophistication as they involved only evaluation of the functions

at a phenomenal number of points. They were able to take advantage of th巴

plentiful computer resources available today. Imagine operating in the

peer-to-peer computing environm巴ntafforded by“MP3" or“Gnutella" and solving the problem using all of the PCs in the world. The second method is the

Branch and Bound method. This is based on partial enumeration where only part

of the combinations are explored. The remaining are pruned from consideration

because they wi11 not determine the solution. This is currently the most popular method for discrete optimization for engineering design. The last method is

Dynamic programming， an elegant approach to optimization problems， but which did not gain favor because it involved significantly larger amounts of

computation than competitive methods even for problems of reasonable size. It is restricted to problems thal require a sequential selection of the design

variables. Today， such resourc巴Iimitationsare disappearing in the world of

powerful PCs and hence lhe method deservcs lo bc revisited.

l司

;;:;11

培

82 OISCRETE OPTIMIZATION TECHNIQUES 325

Standard Format-Dlscrete Optlm/zatlon: The author is not aWllI右 ofa

standard format for山 discreteoptimization problem. It subsumcs恥 formatof lhe

coπωponding continuous relaxation problcm. In this book Ihc fol1owing format fol' the mixed optimization problem is used.

Minimize f(X， Y)，医]n<;[Y]nd

Subjcct to: h(X， Y) = [0]; [hJI

g(X， Y)壬[0]; [g]m

J;SXjS.i;'; ;= 1，2，.. .，nc

(8.6)

(8.7)

(8.8)

(8.9)

Yi E Yd，; [Yd，]p，; i = 1， 2，. . . ， nd (8.10)

X represents lhe set of nc continuous variables. Y rep陀 senlsthe set of nd discrcte variables. f is山eobjective function. h is lhe set of 1 equalily constraInts. g is lhe sel of m inequality constraints. Expression (8.9) rep民:sents出eside cons回 intson each continuous variable. Expression (8.10) expresses出eside constraints on the discrete variables. Each discrele variable Yi must belong to a preestablished set of Pi discrele values Yt/I' If nd = 0，出巴nil is a continuous optimization problem. If 11c = 0，出cnit is a discrete optimization probl巴m.If both are p陀 sent，then it is a Mixed Problem.

Contlnuous Relaxatlon: The continuous relax甜onof lhe mixed optimization problcm (8.6ト(8.10)is identica110 problcm (8励ー(8.10)with Y as a continuous variable 邸t.百lCdiscre飽∞ns位凶nl(8.10) is replac吋 bya continuous side cons回 intfor出e

discrete variables of the form of (8.9).官邸 isnot explicitly developed since (8.6)ー(8.9)and a modified (8.10) is fairly representative of出econtinuous relaxed mathematical model.

Reduced Model: The 問 ducedmodel is important in subsequent discussions. It is 山emodel solved after a part of the set of discrete variables are set at some a¥lowable discret巴va1ues.This removes lhose variables from出eproblem as山巴irvalu巴shave been set. The mathematical model is then defined by the remaining discrete variables ぉ we¥lω 出eoriginal conlinuous variables. If Z(nz) are白eremaining discrete variables白紙need10 be solved，出en

Minimize l(X， Z)， [X]"，; [Z]n， (8.11)

Su同ect10: 五(X，Z) = [0]; [11]/ (8.12)

g(X， z) ~ [0]; [i]m (8.13)

J; S Xj S xj'; i = 1， 2， . . . ， nc (8.14)

Zi e z.t，; [z.t):， (8.15)

326 DISCRETE OPTIMlZATlON

Toble 8.10 Op“mal Volue ofCoououous Voriablex. for Discrete Combloauoo 。fXloodx3

X21x) 0.2200 0.7500 1.7300 2.2判。 2.78叩

0.5 0.9067 1.0833 1.4100 1.5800 1.76∞ 1.5 1.2柑O 1.4167 1.7433 1.9133 2.0933 2.5 1.5733 1.7500 2.0767 2.2467 2.4267 3.5 1.9067 2.0833 2.4100 2.5800 2.7600

Note伽 tin the above， 4， is not a new set. It coπ田 pondsto a reduced Y d.. In

Exhaustive Enumeration， generally it is加 emptyset. AII of the discrete variables紅巳

set to some allowable value.

8.2.1 Exhaustive Enumeration

This is白esimpl巴stof白ediscrete optimization techniqu巴S.It evaluates an optimum solution for all combinations of the discrete variables. The best solution is obtained

by scanning the list of feasible solutions in出eabove investigation for the minimum

value. The total number of evaluations is

/1. = IIp/ (8.16)

If eith巴rnd or Pi (or both) is large，自己n出isrepresents a lot of work. lt also represents

an exponential growth in th巴calculationswith由巳 numberof discrete variables. In a

mixed op也凶zationproblem，出iswould involve nc continuous optimum solution of

the reduced model. If the mathematical model and its ∞mputer calculations are回 sy

to implement， this is not a serious burden. If出巴 mathematicalmodel陀quires

extensive calculationsー for ex創nple，自nite element or finite difference

calculations-then some concem may arise. In such cases，limiting the elements of the discr芭tevariables to smaller c1usters may rep陀 senta useful approach.

Example 8.1:τbe exhaustive enumer司tionofExample 8.1 is available in Ex8_1.m.1

Table 8.1 a shows出巴optimumvalue of the continuous variable x I for every a110wable combination of山 discretevariables X2 and x)・Table8.1 b shows山 corresponding

value of the objective function for each of the cases. Since出isis an unconstrained

problem， no discussion of feasibility is involved. There are 20 values and the best solution for Ex創nple8.1 is

x~ = l.i433; X; = 1.5; x; = 1.73; r = 0.1782 (8.5)

There is some good news though. First， each of the 20 cases is essentially a

s;ngle-lIariable op山凶zationproblem. The number of design variables in出巴original

IFiles 10 be dowllloaded from Ihe web 削除a四 indicalcdby boldfuce白肌SS町iflype.

8.2 OISCRETE OPTIMIZATION TECHNIQUES 327

Toble 8.1b Opumol Value of Objecuve Fuocuoo for Dlscrele Combiootion of Xz aod X3

x/X3 0.2200 0.7500 1.7300 2.2400 2.7800

0.5 1.9107 1.3542 2.7915 4.8060 7.8840 1.5 3.3240 1.3542 0.1782 0.8327 2.4707 2.5 8.0707 4.6875 0.8982 0.1927 0.3907 3.5 16.1507 11.3542 4.9515 2.8860 1.6440

model (3) is reduced by出enumber of discrete variables (2). Model reduction is

involved in enumeration techniques.

Algorithm: E.幼 'austlveEnumeration (A8.1) Step 1. ! = inf， X = [0， 0， .. • ， 0]

For every a1lowable combination of (Y.. Y2，・・・・Y"d)=辛 (Yb)

Solve Optimization Problem (8.11)一(8.15)(Solution X・)

Ifh(X¥Yb) = [0] and Ifg(X.， Yb}三[0]and

Ifj(X¥Yb)<!

Thenf←ft.X¥Yb}

X←X.

Y←Yb

End If

End If

EndIf

EndFor

Example 8.2 iIIustrates白巴 useof algorithm A8.1 with feasibility requirements.

Exmnple 8.2 The Aerodesign Club has initiated fund-raising activiti，回 byselling

pizza during lunch in the student activity center. Over the y回 rs出eyhave collected

也tato identify a mathematical model that will r叫 uce白eircost which directly

tr剖 sla回 tohigher profits. Only two types of pizza wi1I sell-pepperoni (x.)佃 d

cheese (x21. The local pizza chain wi1I deliver a maximum of26 pizzぉ.Factoring into

account wastage and the time available to make the sale出eyarrive at the fol1owing mathematical model:

Minirnize f(x.. X2) = (XI -15)2 + (X2 -15)2

Subject to: 8.(Xh X2): XI + X2 S 26

g2(X.・x21: x. + 2.5x2 S 37

(8.17)

(8.18a)

(8. I 8b)

328 DISCRETE OPTIMlZAnON

XI ~ 0; x2 ~ O. and integer (8.18c)

The graphical solution for continuous relaxalion is available in Figure 8.2 and the

analytical Solulion (not included) identifies that 12.86 pepperoni pizzas and 9.65 cheese pizzas will provide for the minirnum cost of operation.

Ex8_2.m is山em-file出atwill perform the exhaustive enumeration of the problem. The discrete solution is

X; = 12; xi = 10; r = 34; g2 is active (8.19)

This confirms伽 tneighboring discrete values around a discrele solution works well. Programming exhauslive enumeration is straightforward and particularly easy with MATLAB. Given出巴processingspeed， large memory availability on白edesktop.国 sy

programming through software like MATI..AB. e対祖国tiveenumeration is probably a

very good idea loday. Whal recommends it even further is白紙thesolution is a global ゆtimum.FlIrthermore. unlike the earlier chapt巴rswhere th巴conceptsof derivatives. matrices. linear independence. and so on were essentia1 to understand and implement the techniques of optimization， lack of sucb knowledge is no disadvan包geto solving a discrete optimization problem using exbaustive enumeration. Tbere is only one essentiaI skillー凶Is1ating恥 mathematicalmodel into program code. With出Eexperience provided in出isbook. this is definitely not an issue.

40

35

‘ー、 g~=O.. ‘-....

・ ¥6ie-7・~、二

5 10 15 20 25 30 35 40 pepperonl plzza

Flgure8.2 G阻phicalsolutlon Example 8.2.

時ヨ;

8.2 DISCRETE OPTIMIZATION TECHNIQUES 329

百 esolution in Equation (8.19) can be obtained with continuous rel蹴叫onof山E

problem. In this case the discrete solution has to be recovered by adjusting the

continuous solution to neighboring discrete values.

Exhaust#ve Enumeratlon for Contlnuous Optlmlzatlon Problems: This title

must be quaIified further-in lieu of gradient-bas巴dnumericaI techniques. The

availability of serious desktop computing power. the promise of global optimization.

and由巳easeof implementation suggesl a multistage exhaustive enumeration strategy for continuous optimi回 tionproblems. Exhaustive enumeration after all is a search strategy. similar 10 the ones used before. The search for solution is akin 10白e

zero-order melhods employed for unconstrained optimization (Chapter 6). The higher stages in multistage enumeration involve enumerating on decreased

intervaIs centered around白esolution from白巴 previousstage. The vaIues of the

design variables used for enumeration will incorporate progressively improved

tolerance. The strategy must take into account the possibility that the initiaI interval mighl overlook globaI optimal solutions tllat do not appear attractive in a particular

stage. Hence. more than one minimum is tracked in each stage. Each of these

minimum can be considered姐 enllmerationcluster. Each such c1uster is multistaged until a prescribed tolerance is achieved. In engin田 ring.m阻 ufacturingtolerance of the order of 0.0001 is achievable today. Tolerances below出islimit probably need to be justified unless白earea is microelectronics. From a programming perspective it jllst

adds an extra loop to the implementa1ion.百lefinal solution obtained represents a

continuous solution within the final lolerance implemented by the technique.

Such an exercise is lefl10 the student. As with all implementation the merit of lhis

approach lies in the tractabili1y of the mathematical model and the need for a global

optimum solution. Since the method is largely heuristic， there is no guarantee that the solution is actuaIly a global optimum (of course assuming the functions involved are

not convex).

8.2.2 Branch and Bound

Exhaustive or complete enumeration is possible only for a limited set of variables

because of the exponentiaI growth of effort問qui民 dto examine aIl possible solutions. Using partial (or incomplete or selective) enumeralion to arrive al the solution to the discrele optimization problem would provide tlt巴 necessaryadvantage in handling

large problems. Partial enumeration works by making decisions about groups of solutions rather出叩 everysingl巴one.The 8ranch and 80und (88) algorithm is one

of the success向Ialgoritl1ms to use selective/partial enumeration. lt does so by using relaxation models instead of the original comple1e discrele ma1hematicaI model. The

8B algorithm uses a tree structure 10 represent cqmpletions of partial solutions. The 町'ees汀uctureuses nodes and edges/links 10 represenllhe trail of partial solutions. This is call吋 fatlromulg出ep釘tialsolution. The terminology is borrowed from operation

rese邸 hliterature [3] and is explained below wi山 resp削 toExample 8.1 which is

reproduced here for convenience:


Minimize f(Xl・X2・X3)= (XI -2)2 + (XI -X2)2 + (XI -X3)2 + (X2 -X))2 (8.1)

XI E R (8.2a)

X2ε[0.5 1.5 2.5 3.5] E X2d (8.2b)

X3 E [0.22 0.75 l.73 2.24 2.78]ε X3d (8.2c)

Partlal Solutlon: A solution in which some discrete design variables are fixed and o山ersarejト'eeor undetennined. Variab1es that are not fixed are represented by出esymbo1 γ:" For example，

x = u: 0.5，刀 (8.20)

rep問 sentsthe optimization prob1em wi出X2= 0.5 and XI and X3 to be determined. The pa凶alsolutions ar芭a1sotermed the nodes of the tree.百lefree variables are usually solved using a continuous rel回出onof the model.

Compler/ons (0' PaTt刷 Solutlon):Ench partial solution or nod巴cangivt: rist: lu additionnl partia1 solutions/nodes directIy under it. This is called brallching or expansioTl of lhe node. In this new set of nodes another one of出巴 frecdiscrelc variablcs is fixed and白eremainder slill continue to be free. For example， under出巴

node represented by the partial solution in Equation (8.20)， five additiona1 nodes are 卯ssible:

X = [f， 0.5， 0.22]; X = [f， 0.5， 0.75]; X = [f， 0.5， 1.73];

X = [f， 0.5， 2.241; X = [f， 0.5， 2.781

Thesc completions branch out and hcnce a treelikc structure. There is a hierarchical structuring of the nodes. Each branch represents 1eads to a tier in which some variable is held at its prescribed discrete value.

Fathoming the Partial Solutlon: Expanding the node all the way till the end is termed fatlwming a node. During the fathoming operation it is possible 10 idenlify those partial solutions at a node that do not need to be investigated further. Only the best solution at 山e node is branched or completed. Those part討ialsoωlu凶』此川t“io叩n凶sll凶Inoωoωdeωs山a瓜t

will not b恥einvestigated further are pruned 0町rtenninated i加n白elI出芭C.

Actlve Partlal Solution: A node thal has not been tenninated or pruned and has nol bcen completed.

EdgeslLlnks: Thes巴釘'enode-to・nodeconnections in a町田.They identify the value _r ・1..・_..~.. ..n_備 L・司'.......:例目........J

8.2 D1SCRETE OPTIMIZATION TECHNIQUES 331

Root: Th巴BBalgorithm searches a10ng a II芭C.lt starlS al the root where a11 the

部23t九弘:31?お1J訪問:叩:;;:??J22::preferred to jumping across nodes in白esame凶erarchy.

Incumbent softJMont The incumbent solution is thsbGSt feasible solution Mown SO far with a11 discrete variables assuming discrete values. When lh巴BBsearch final1y stops，白EhalinCUREbBEEtsolution，ifomexists，is a ziobal optimum soitHion-h thefollowing theincumbentsolution is denotedbyr--13mincumbaudESign vmables眠 X...

Example 8.1-Uslng Branch and Bound: Here血eBB search is appliedω Example 8.1.BBEX8JJn calculates allof thE data needed for genEraIing血巴住民・

TheBB町田itse1fis developed in Figu民s8.3 and 8.4.

Start: Node O. All of出巴variab1esare f民C.The solution to the continuous relaxation ofthe obj田 tivefunction (solved as a continuous problem) is identified at the

node. Nod巴s1-4: This is th巴firsttier of the BB tre巴・ Thediscrete variable X2 is assigned

Each of thgpennisBiblediscrete values at the nudcs in this tier-This ES indicated

X=[f，f，fJ

X* = [1.74，1.5， 1.73] p =0.178 p* = 0.178lncumbenl solutlon

XoI". = [1.74， 1.5， 1.73J

Flaure 8.3 Branch and bound Example 8.1.

332 DlSCRETE OPnMIZATION

④ X" = [1.57， 2.5， 0.22]

P=8.07

X" = [1.75，2.5，0.751 P=4.68 xe=I107，zs，1.731 x'=I2.26，zaz.Ml

P = 0.898 P = 0.192

F1gure 8.4 Branch and b司undExample 8.1・-fathomingNode 3.

on the edges between the nodes in Figure 8.3. The remaining v釘 iables鉱B

fr，田Icontinuous.The optimum solution at the various nodes is displayed on出E

tree. The minimum solution is the one出atwill be fathomed at the next step.

Here Nodes 2 and 3創-ecandidates for expanding the branches. Nodes 1 and 4

will not be fathomed (unlcss their optimum value is better出anIhe incumbent

solution)一山eywill be terminated or pruned脳血eobjective function has a

higher valuc.ηle continuous relaxation ensures a lower bound on the optimum

value of the objective at the node. Node 2 is taken up for fathoming based on node ordering

Completions of Node 2: This expands Node 2 by setting血ediscrete variable X3 10

its permissible set of discrete values. It creates branches Nodes 5-9. lt is also

the second tier of the tree. The solution is indicated at出evarious nodes. In all

of them the two disc問tevariables are set 10 indicaled values while Xl is free.

which is a continuous variable by definition. Since all of the discrele variables

bave discrete values， this node is also fathomed. Node 7 indicates the best

solution. Since all of出edisc民tevariables arち assigneddisc悶 levalues，出E

solution at lhis node is the current incumbent solution・Theremaining nodes will be pruncd. The currenl incumbent solution (f. = 0.1782) is higher白叩由巳

solution al node 3 (f = 0.15). This requires Nod巴310 be fathomed.

Completions ofNode 3: Figure 8.4 iIlustrates the branching at Node 3. The process is similar 10 completions of Node 2 above. The best solution is at Node 13 with

a value off = 0.1927. This is largerthan the Current incumbent solution. AlI of 出enodes can be lerminaled.

No further nodes are available for expansion. The current incumbent solution is

also the global optimal solution. This is the solution to山eoptimization problem. In

8.2 DlSCRETE OPnMIZATlON TECHNIQUES 333

出isexample， 15 partial solutions were n民 essary.Each partial solution can be

considered an enumeration. In exhaustive enumeration， 20巴numerationswere

required for Example 8.1. There is a nel gain of 5 enumeralions. Th巴 BBsearch redu田 sthe number of total enumerations need巴d10 solve出eproblem.

Algorlthm: Branch and Bound伺8.2):For even a small set of discrete variables actually drawing th巴 BBtree struclure is most likely a difficult enterprise. The 11'・eestructure merely idenlifies the search process. Example 8.1 did not have constraints so feasibility was nol examined. With constraints present， choosing the p釘 tialsolution to complete based only on minimum value of the objective function is

not a justifiable s回 tegy.In this case the besl feasible solution is山ecandidate for completion. Also， an active feasible partial solution白紙 hasa better value出朗自己

current incumbent solution is also a candidate for completion. The following

algorithm capωres白巴essenceof the Branch and Bound algorithm.

Step O. Establish the root node with all discrete v訂 iables合問.

lnitialize solution index s to O. .. Sel current incumbent soIution to f--= ""， X--= [∞]， (minirnization

problem).

Stcp 1. If nny active feasible parua1 solutions remain (白紙isbeuer than the current incumbenl solution)， selecl one as X<J) and go to Step 2.

If none exists and there exists an incumbent solution il is optima1.

If none exists and the陀 isno incumbent solution the model is infeasible.

Step 2. Obtain白ecompletion of the partial solution Xωusing a continuous

relaxation of the original model.

Step 3. If there釘eno feasible completions of白巴 partialsolution Xω， terminate X<I)， increment s ←s + 1， and retum to Step 1.

Step 4. If the best feasible completion of partial solution X<I) cannot improve the incumbent solution， then terrninate Xω， increment s ←s + 1， and retum to Step

Step 5. If白ebest feasible completion is better由加 theincumbent， update the

incumbent solution

r.←!<I)， x..←xω

terminate X<s>， increment s ←s + 1， and retum to Step 1.

Example 8.2 The Branch and Bound algorithm is applied to Example 8.2. The

S国民his applied through the BB住民 . 百lemathematica1 model for Example 8.2，

whereXl・andX2 are integers， is

Minimize !(x" xv = (Xl -15)2 + (x2 -15)2 (8.17)

334 DISCAETE OPTIMIZATION

Subject 10: 81(X" X2): X( + x2 :!> 26

82(XI・X2): X( + 2.5x2 :!> 37

XI ~ 0; X2 2! 0

The solution is

(8.18a)

(8.18b)

(8.18c)

X;= 12; X;= 10; f・=34; g2 is active (8.19)

ヱiz:;22J31f尽なe?Litzは522eifz::i::zztszfollowing development

X1 E [10 Il 12 13]

ろE [9 10 11 ]2)

Uslng Branch and Bound:百 eBB田 eis shown in Figure 8.5. The data for曲e町田町C必cuJat吋 throughBBExι2.m where the continuous陀 l蹴 ationis solv吋analyticaIly using thc KT conditions.τ'he tree is brnnched ns follows:

XCo:::[13，9) 1'*=40

X. = [12.86， 9.951

1'* :::33.13

1nfeaslble 1nf伺 slble

Rgure 8.5 Branch and bound Ir自白forExample 8.2.

Infωslble

8.2 DISCAETE OPTIMIZATION TECHNIQUES 335

NodeO:百出isthe root node. All the variables釘efr，田.百econtinuous relaxation

of the problem is solved. The solution is indicated on由巳tree.Currently this is the only available active partial solution and hence needs to be completed.

Nodes ]-4:百epartial solution at Node 0 is completed using variable XI from the reduced discrete set identified above. This leads to four branches and the optimal values ar芭 indicatedon the紅白.The best values are indicated at出enodes. For血isproblem出巴.yare also feasible. There紅 enow four active partial solutions.

Node 3: The so]ution at Node 3 is also a feasible solution of the original discrete optimization problem since the design variables are discrele and belong to白epermissible discrete set. Node 3 isfatJlOmed.百 isprovides the first incumbent solution.

Th巴reare three available active feasible partial solutions (Nodes 1，2， and 4) for completion. Since the solutions at白enodes are all feasible， only the best active feasible partial solution is picked up for completion-Node 4. Nodes 1 and 2 can be terminated.

Completion Node 4: Node 4 has a feasible solution better曲朗自ecurrent incumbentsolution.ltis completed by branching 10 Nodes 5， 6， 7，8. Only Node 5 has a feasib]e solution. Tbe besl solution al Node 5 is not better than the current incumbent solution. Nodes 5， 6， 7， 8 can be terminnted.

There are no more active pa凶alsolutions and出巴加ehas becn fathomed. The current incumbent solution is the optimal solution.

百liscompletes the BB search applied to Example 8.2. The 陀ductionin enumeration， compared to由巴exhaustiveenumeration， is difficult 10 comment on for two reasons.

1. The deliberate choice to restrlct白ese1 of discrete values used 10 branch the 紅白

seemed arbitrary. While白iswas done for convenience， a serious justification 伺 nbe advanced based on the continuous solution at Node O. The reduced discrete set is chosen to include a limi1ed neighborhood/window of the continuous叩 1凶 on.

2. The exhaustive enumeration， based on disc陀tevalues， was easierto evaluate for a pure discrete problem. Applying KT conditions to obtain the possib]e p紅tialsolutions to出econtinuous relaxation， and choosing the acceptable solution is an involved exercise. Using numerical lechniques Iike SQP 10 solve the problem， which wil1 mosl Iikely be done in practice， has its own problems. Dependence on good starting values wi出noguarantee of convergence requires user inlervention 10 identify血eparticular solution. It would requir芭 alot of effort 10 automate the pro白 5Sso as 10 apply山eBranch and Bound algorithm 10 general nonlinear mixed optimization problems. Linear mixed optimization problems do not possess血islimitation， at least to the same extent.


In山isexample，白eactive partial solutions at the root and in山巴 firsttier were all feasible. What would be the strategy if the root nod巴orother nodes have an infeasible active partial solution? In白紙 case山e88 search process must complete all active partial solutions and not only feasible active partial solutions. That is a modification 山atcan be easily made to the 8ranch and 80und Algorithm above.

Other Search Techniques:ηle two search techniques in山issection， n創nely，exhaustive enumer冨tion，and branch and bound， provide an additional feature of discovering global optimum solutions.τ'hey also belong to the c1ass of methods used to address global optimum.τbe回 syavailability of large and faster computing resources has brought renewed emphasis to globally optimal designs. SimuUlted annealing and genetic algorithms ar芭 amongthe leading candidates for global optimi祖 tion.百ley釘 ecovered in the next chapter and can also be characterized as enumer混ion飽chniques，like出巴onesin白issection. 80th of them can and are used for discrete optimization. They釘'enot discussed in this chapter. The application of most of these techniques is still heuristic and often depends on experience based on numerical experiments among c1ass巴sof problems. Some applications are problem specific and require user intervention and learning. Standard implementations in由esemethods. especially for nonlinear problems， are nol yet the norm.

8.2.3 Dynamic Programmlng

Discrete dynamic programming (DDP) is an exciting technique for handling a special c¥ass of problems. Examples 8.1叩 d8.2 are not direct members of this c¥ass.則chard8ellman [4) was responsible for伽 tin加 ducing出econcept and the algori山m.It is an op白凶zingproαdure based on Bellman・sprinciple of optimality. The principle is based on a sequen田 ofdecisions on partial solutions. so曲atwhen ccrtain variables have been determined. the remaining variables establish an optimum completion of the problem [5). Ano出erexpression of this戸ncipleis ob包inedfrom Reference 6. An optimal sequenα of decisions h脳血eproperty that at出ec町renttimc. whatever thc initial state and白eprevious decisions.山E陀 mai凶ngdecisions must be an optimal set with regard to血estatem 刷 ngfrom the first d出 sion.DDP is about a sequence of decisions (often in time) and is sometimes term吋 asequential ，たdsionproblem. Problems Iike Examples 8.1叩 d8.2do not directly fit白iscl俗sifi回 tionbut they回 nb山田sformedto fit白isr珂uirement(5). Very often. DDP problerns can be described as卸 optimalpa出 determinationproblem. Such P瓜hproblems are solved using directed graphs (digraphs).百leseproblerns may actual1y have no connection to physi回 Jpaths or distances. Like the 88 tree，出巴ycnablebetter appreciationlapplication of the algorithmic proωdure and are not s汀ictly陀quired.DDPr珂u田sdefi凶tionof s.ωtes and stages. and only the former is discussed he隠.百leequationsωed to es包blishthe principle of optimality are回 ))edρUlctiollalequations.

Example 8.3 will be used to define血enewterms.

Example 8.3 An established university is inl巴胞stedin developing a new Co))巴geforInformation Technology during the currenl ye釘. Table 8.2 ilIustrates恥 data

自，2 DISCRETE OPTIMIZATION TECHNIQUES 337

Table 8.2 Faculty H1ring Pollcy Data for Example 8.3

ltem Fall Winter Spring Summer

R明uiredFaculty 5 10 8 2 Recrui回 entCost 10 10 10 10 Unit Facu1ty Cost 2 2 UnitRe凶ningCost 2 2 2 2

necessary to calculale the cost of recruiting faculty for山isenterprise for the first ye釘of oper富山n.The college will operate during four quarters (FaU. Winter. Sp由 g.組 dSummer) indicated by出ecolumns. Decisions are associated with the beginning of the quarters. The first row indicates the new faculty required to implement the cours白血atwill be offered. The second row is山erecruitment cost for new faculty each quarter. which is significant because of山ecompetitive d巴mandfor qualified people in this 釘回.百le白irdrow is出eunit cost of new faculty in terms of the standard institut巴faculty cost. Course speciali回凶onand availability makes this cost different in different quarters. The last row is出巴 unitcosl for retaining extra faculty hired in出Eearlier quarter (to minimize cost over the year). What should be the hiring policy for 由巳universilylo minimize cosl?

百lisexample is c1early il1ustrative of sequential decision making. Decisions are requ耐dto be made every qu副 er.For example， it is possible 10早cruitall needed faculty during出efirst quarter and pay出B問凶ningcost. or it may cost less to hire during回 chquarter， or~tforfirst 血reequarters and hire again in白efo町出.百leproblem will be solved using加 optimaUshortestpath method even曲oughthe隠 arenorou回 involvedin出.eproblem.τbebest ways to understand白eshortestp砥halgor抽 mis to draw a directed graph (digraph) betw田 nthe nodeslstates in the problem.

S旬tes:States問presentpointslconditions in DP where decisions have to be considered. These are sometimes児島π'edto as policy decisions or just policiω. This has a s叫uentiaJconnection as current decisions cannot be∞nside陀dUD世出eprevious decisions have been made. Clearly in Example 8.3出町民present白ebeginning of each quarter.明lenumber of states is usually one more由加 thenumberofd巴cisionsto incorporate the effect of the last policy/decision. Nodes in the digraph rep陀 sentstates in the DP problem. The digraph represents a connection between each pair of states出atis permissible. This can be made c1earer by stating伽 tif a node is going to be affected by a decision at any previous nod巴.then there is a direct connection (Iinelarc) between山enode喧.The cost of the connection is indicated on由Econnection itself. An arrow can be used to indicate the direction of traverse along血islinelarc segment. These Iin巴s/arcs由ereforerepresent decisiofls. Once the digraph for the DP problem is available. the optimum solution correspoflds to the shortest path from Ihe beginnillg 10 the ending state ofthe digraph [3).

338 DISC円ETEOPTIMIZATION

99

74

Flgure 8.6 Digraph for Example 6.3.

Figur巴8.6is由巴 digraphfor the hiring policy. There are five states with the first

four representing出equ紅白隠 wherethe decisions must be made. Node 1問P陀sentsthe Fall quarter and 50 on. Along the arcs on山edigraph is th巴C05tof the deci5ion or policy出atwiJI originate at the current node and meet a11 of the requirements at曲e

l刻印rnode. In order 10 us巴theinformation in the digraph effi巴clively，and 10 develop Ihe mathematical model， the quarters are assigned indices from 1 to 4. The following

abbreviations are used: required facullY， RF;陀 cruitmentCOSI， RC; unit facully cost，

FC; unil relaining cost -holding cosl， HC. The policy decisions or cost between the start node i and the end nodej is C(i，j). For example. C(l， 4) rep問sents出epolicyめ

副民泊 theFall q凶 rter，which will meet出estaffing req凶rementsfor出eWinlerand

Spring quarters a150 thus paying曲eappropriate retaining pena1ties:

C(l， 4) = RC(l) + [RF(l) + RF(2) + RF(3)).FC(l)

+ ([RF(2) + RF(3))巾HC(l)+ [RF(3)]吻HC(2))

Ex8_3.m is the m-file白atgenerates the data for the digraph for Exarnple 8.3釦 didentifies山e50lution by solving thefunclional eq凶 lionsand backtracking山epa白

to identify the optima1 s巴quence.Note出atfor differ芭ntproblems on1y the cost calculation wiJI differ. The functional equalions and backtracking紅巴 elementsof the DDP method and should be the same.

Functional Equatlon for Dynamlc Programmlng: The functiona1 equation is the essential part of the DDP a1gorithm or method. It employs a recursive pro印 du児

10 identify the besl solution al each state. Lel Value (i) be th巴bestvalue of the decision

al node i. laking inlo account a11 possible arcs/paths thal lead to node i. Let Node (i) be the node白紙ledto the best policy at白ecurrent state. The recursion relations for IJ states can be sel up as follows:

8.2 DISCRETE OPTIMIZATION TECHNIQUES 339

Value (1) = 0 Value (k) = min (Value (の+C(/， k); 1 S 1 < k}， k = 2， . . . ， 11 + 1

Forexampl巴， in Example 8.3

VaLue (4) = min (Value (1) + C(l， 4); Value (2) + C(2， 4); Value (3) + C(3， 4)}

Value (4) = min ([0 +85]; [15 + 62]; [45 + 26]) = 71

Node (4) = 3 (which establish巴dthe minimum value)

These ca1culations are embedded in Ex8_3.m. The output from the program IS:

Example 8. 3-Dynamic programming Cost between pairs of nodes/states C (i，j )

o 15 45 85 99

o 0 30 62 74 o 0 0 26 34 o 0 0 0 12

00000 odes -Values (i) optimal path values at var~o

o 15 45 71 79 Best previous nodeヒo reach current node -Node(i)

0123 3

The Value (4) is 71 alld Node (4) is 3. The optimum value for the problem is Value

(5)， which is叫1Ua110 79.

Shortest Path: The shortest path is obtained by backtracking from tlle end stat巴

凶 ingthe Node (i)infonnation-which is the row of data above. At NodB5thEbest nodEfrom which to reach Node 5is Node3

o 233

AINod巴3the best node from which to reach Node 3 is Node 2

o 233

At Node 2 the best node from which to reach Node 2 is Node 1 (this is山estartmg

node)

o 1 233

This solution is indicated by出ethick soIid line in Figure 8.6.


τ百ne臼ac伽加a叫Ih刷li山耐i討凶ri巾t訂加n略Igp卯olic必cya釦.cco∞orl凶d伽n昭gtωo白iぬss叫olu凶tio∞niおstωohi甘i町r隠e叩s抗tt白her問巴叫q伊u凶山iI1凶edf:批a配c叩u凶山It旬y for血巴 FaJlqu釘 t総er泊 theFaJl qu釘 t匂er.Do the same for the Winter quarter. For the Spring quarter， however， hi陀 forboth the Spring and Summer quarters.

Alternate Paths: There is an alternate solution for thc same optimal value. The output from Ex8_3.m includes出isinformation though it was not included above. In 曲eoutput in the command window， each row contains additional nodes出atwill yield

same optimal vaJue.百leonlyen町yis 1 in third row and second colurnn. This suggests

that 鉱山ethiI1d node， Node (3) can山 ohave the value of l-that is， Node 1 also provides the same vaJue to reach Node 3. This will cause出eNode (i) information to change as

。 3 3

The shortest pa出 C朗自enbe interpreted as follows:

At Node 5 the best node from which to reach Node 5 is Node 3

。 3 3

At Node 3 the best node from which to reach Node 3 is Node 1

o 1 3 3

In Figure 8.6 this is indicated by the thick dashed arcs. The hiring policy now would be to hire for FaU and Winter in Fal1. Then hire for Spring and Summer in Spring.

~/gorlthm: Dynamlc Pro，吻rammlng伺a勾The fol1owing algorithm is identificd in-Refcrencc 3 for pa山sthat do not cycle. The

original 8el1man algorithm incJudes an iterative 陀 finementthat allows for cyclic

pa血s.In Example 8.3血ereare no cyclic pa白s.This algorithm is implemented in Ex8_3.m.

Step 0; Number nodes in sequential order. Arcs ar芭 definedfor j > i. Set the start node S optimal value as

Value (s) =0

Step 1: Terminate if all Value (k) have been calculated. If p represents the last unprocess巴dnode. then

Vallle (P) = min{ Va/ue (i) + C (i. p): if arc (i， p) exists 1 S i < p}

Node (p) = the number of Node i that achieved the minimum above

Return to Step 1

8.3 AOOITIONAL EXAMPLES 341

This was a simple example of DDP. It is important to note白atsolution in final

均 lue(りincJudeson1y the best solution at all states/nodes. Like other techniques， the偲創'cseveral different categories of problems addressed by DDP. incJuding special

discrete problems like the Knapsack problems. It is also possibl巴tobring Examples

8.1 and 8.2 under the framework of DDP. though to impart a sequenti凶naturl巴inthe

process of selecting design variables in this case may involve a to凶 transformationof

白eproblem. Even then it may not be efficient to solve it using DDP. In practice， DDP has always becn regarded as computationally complex and prohibitive. It is seldom used for problems wi出 alarge number of states， or if the estimation of cost is intensive. Another criticism of DDP is白atit ca而esout a lot of redundant computation for continuous problems. Today. these are less of an issue山肌 inthe past. For stochastic and discrete problems DDP may sti11 be an effective tool. 8el1man has also indicated that it c朗 beuscd to solve a set of unstable systems of equations.

DDP also compu也sdesign sensitivity as part of its solution. This section was

prim訂日ymeant to introduce DDP as a useful tool for some categories of problems.

Readers arc encouraged to consult山ereferences for more information on白esu句ect.


In this section. two additional examples are presented. The first one is an engineering cx創nplefrom mechanical design und illustrates how the disc問 teversion of a four-variable problem very effectively reduces to a single-variable problem白atcanbe easily ad也国sedby enumemtion. An unbelievable reduction in complexity

yielding global optimization is obtained by elementary calculations. It definitely

invites scrutiny regaroing出巴 E仔ortin solving continuous nonlinear optimization problems. The second one is出eapplication of the bmnch and bound search to a 0-1 intcgcr progmmming problem. Much of the application of discrete optimization is in

也e釘'eaof decision ma話ng.and decisions like for or against. yes or no， are possiblc using a 0-1 linear integer programming model.

8.3.1 Example 8.牛ーIBeam Deslgn

τhis cxample is a structura1 design problem. In facしitis another variationlapplication of the I-bcam structure seen回 rlier.The continuous problcm was developed and solved as part of a student project. In山isexample出ebeam is used as a horizontal

member in a fixed“E" series gan町 cmne.The hoist， with a capacity of W = 2000 Ib. is at the center of the be阻 1.The objective isωdesign a minimum mass be創nthat will

not fai1 under bending. shcar， and spccified detlection. The length L of the beam is

specificd as 96 in. Steel was choscn for thc material of出ebe創n.Thc modulus of

elasticity of steel is E = 29 E+06 psi. The weight density of s同 Iis P = 0.284Ib/in3•

The yicJd s田 ng山 in健闘onis σ'y = 36 E+03 psi. The yield streng出inshear is τy= 21 E+03 psi.百esp配 ifiedallowable detl民 tiODis 0.25 in. A factor of safety of 1.4 on

the load is assumed. Figure 8.7 iIIus回 testhe cen回cloading on the beam as well as

the design variables. (A11 calculations are in inches and pounds.)

/

/

.:....:<<'1 x

/ /

/

343 8.3 ADDITIONAL EXAMPLES DISCRETE OpnMIZAnON

L ド

342

The Dlscrete Problem: The discrete problem shares the same mathematical model except the values for the design variabl巴swill belong to a set of discrete values. It is possible to use巴ith紅白巴 completeenumeration or the BB search to

solve由巳 problem.Before proceeding further in this direction， it is useful to

recognize白紙 thereason for discrete optimization is to choose an off-the-shelf

I-beam which will keep the cost and production time down. Several mills provide information on standard rolling stock曲eymanufacture. Reference [7] is used to

collect data for this example. lf a standard S-beam is needed，出ereference lists

several beams with acωal values for the design variables XI・X2・X3・X4・Choosingan I-beam from the list also identifies all of the design variables. This becomes a

single-variable problem， the variable being the particular item from the list of

beams. In this case there are no geometrical constraints necessary and neither are

出eside constraints. A complete enumeration can be performed on the candidale

beams from出巴stocklist， and the best beam can easily be identified. Compare this enumeration to solving the nonlinear mathematical model with scven functional

constraints. When such an opportunity arises， exploiting the discretc optimization problcm is highly rccommendcd. On the other hand， consider a different problem where a plastic I-beam wi1I be molded in the manufacturing faci1ity. Hcre the

continuous problem needs to be solved for the minimum cost. Ex8_ 4.m wiIl perform a complete enumeration of this example. It includes a

truncated stock list from which the fmal solution is obtained. The programming is

straightforward. The output in the Command window is

X:=Oお 1"=48.239x; =0.1563; X;=2.∞; X~ =5泊 28;

Optimum Beam is :W6X9 Mass of the Beam 71.543 (lb) Depth， Width， Web Thickness， Flange Thickness

5.900 3.940 0.170 0.215 Constraints: Deflection， Normal

(<= 0)

-2422048221.600

Solution found

(inches)

Stress Shear Stress，

-86086.261

Note that山esolution is significantly higher白anthe one obtained in the continuous

relaxation， once again illus凶 ting白紙 thecontinuous relaxation provides a lower

bound on the solution.

v-MY

'til

ZX4÷

(normal slress)

凶1

コ111

(defl配 tion)

Fド」

The mathematical model for this problem can be exp陀 ssedas:

Minimize

Subject to:

Design variables: Example 8.4.

f.i.X) =pLA.，

gl(X): WU -12EI~ 0

g2(X): Wlxl - 8/，σy~O

g3(X): W仏-21X3τyS0

Figure8.7

(shear s悦 5S)

Some geomctric constraims to relatc the various design variables ar巴

g4(X): xl -3X2 ::;; 0

g5(X): 2x2 -xl ::;; 0

g6(X): x3 -1.5x4 ~ 0

g7(X): 0.5X4 -x3 ~ 0

The side constraints on由edesign variables are

0.25 ::;X4::; 1.25 0.125 ::;x3 S 0.75， 2::;x2::;15， 3::;x1 S20，

-2468680.570 The following relations defme出ecross-sectional a悶 a(Ac)， moment of inertiaの，andfirst moment of area about the cen回 id(Qc):

Ac =2xr4 +X1X) -2勾X4

1 = X~ ~ _ (x2 -X4)(X1 -2x3)3 = ----=---'--"'------::

12 12 Zerか心neInteger Programmlng

Many LP programs/models have special features which are usually exploited tbrough

special algorithms. An important class of LP problems require出evariables be binary， that is，出巴ycan have one of two values. These problems can be set up so白紙 values

8.3.2

Qc =0.5.ザiXI-X~+0.5x3(XI -X~2

Thc solution to the continuous relaxation model is

345 8.3 ADDITIONAL EXAMPLES

調:

DISC阿ETEOPTIMIZATlON 344

Capi組1Budgeting Example Table8.3

VaJue Year3 Year宮Yearl Group Item

AuqJAUZJhueJ

内

&

弓

3

'

a

nunu-Enu'A内JU

-E『M

司

3弓

3nU44

321250

Systems Microsystems Software Modeling M脂血IsPhotonics

‘a内，“司

3必

ιTZJZO

3

X" • • • ，X6ε[Oor 1]

。。く-4

0

x

+

a守x

今コ+

3

3

〈】

ι叫

勾

ー一

Z

+

5

b吋

+

+

判

勾

8 10 Budget

i= 1， 2，... ，m ιu g

--J X a

nT'MMF

Minimize

Subject to:

Note白isis not a minimization problem as requi問dby the standard format.

A句orithms:百.ereare severa1 algo抽 ms伽.1have been developed for solving ZIP models. They all use implicitJselective enumeration白roughBranch and Bound s朗氏h，together with continuous relaxation. However， the binary nature of出evariables is exploited when completing the partial solutions or回 versingthe trl田.百leBalas Zero-One Additive AlgorithmlMethod is one of脈問commend吋臨加iquesfor出sc1ass of problems [6]. The algorithm from出巴samereference is inc1uded below.

Xj = 0 or 1， j = 1， 2， . . . ，n

Except for the binary sid巴constraintson出edesign variables，出isrepresents出emathematical model for an LP problem (the standard model [1叫凶resequality cons回 intsonly). Most algorithms assume出巴今M芭positive.If any cj is negative， then 1一巧issllbstitllted for xJ in出eproblem.

Several varieties of optimizationJdecision problems involve binary variables. They inc1ude the knapsack (a pure ILP with a single main constraint)， capital blldgeting (a multidimensional knapsack problem)， assembly line balancing， matching， set covering and facility location， and so on. Some examples are detailed below.

for白evariables are zero or one--defining a Zero-One Integer Programming Problem (ZIP). The general form of a ZIP is

f(X): :2，明}=1

Example 0' a Knapsack Problem: Minimize血enllmber of pennies (X1)， nickels (x21， dimes (X3)， and quarters (X4) to provide correct change for b cents:

X1 +x2 +x3 +X4

X1 + 5X2 + IOx3 + 25x4 = b

創'emtegers

Minimize

Sllbjectω:

X" X2，X3，X4

In白eabove problem， the unknown variables are not binary variables but釘 eintegers none白eless.

A匂orl肋 m:Balas Zero-One Additive A匂o.r~~h!!， (A8.4) The enumeration is佐ackedby two vectors U aitd X. Both紅巳 oflength n. lf a partial solution consists of巧1，ろも・..，巧sassigned in order， the values of X corresponding to白eassigned variabl巴sareOor 1.τ'he remaining v紅 iabl回紅e合'eeand have the vallle -1. The componenωof U are

λ Ii江f巧k=J a叩ndi出tωsc∞om岬pμIlem巴叩n飢Itけhasno悦tb巴民c叩nc∞o叩nω ere II句llk=イ古叫:k~if ぅ= 1 or 0 and its 叩 pJem側 h 叫 een consid 町 d

o I ifk>s

Step 1. If b :2: 0， optimal sol凶 onis X = [0]. Otherwis巴U= [0]， X = [ーJ].f.= 00，

X. = X (incumbent solution)

Step 2. Calclllate

Example 0' Capital Budgeting:百leCollege of Engineering is considering developing a unique Ph.D. prograrn in Microsystems in the next thrl田 years.This expansion calls for significant investment in resources and personnel. Six independent focus areas have been identified for consideration， a11 of which c佃 notbe developed. Table 8.3 pI1CSents the yearly cost associated with creating the various grOllpS over the 由民ey回 rs.百lelast line in出etable represents the available blldget (in millions of dollars) per year allotted to this development.百lelast column is出evallle (in $100，000) the group is expected to generate in the next 5 years.

The capital budgeting problem llses six binary variables (Xh . • • ， X6)， one for each item in出etable. The ZIP problem can be fo口nulatedas

J is the set of assigned variables y，=bj- :2，a内，

20X1 + 3X2 + 30X3 + 5X4 + IOxs + 5X6

3X1 + 2x2 +x3 + 2x4 + 5xs S 10

Maximize

Subject to:

346 OISCRETE OPTIMほATION

y = min Yi' i = 1， 2， • . . ， m

I=LCfj

Ify注O加 d1<1' then I=J， and X・=X Go to Step 6

0出erwisecontinue

Step 3. Create a subset T of合-eevariables巧definedas

T=ij:f+今<1，aij < 0 for i such曲aty/<O)

1fT = o (empty set) then Go to Step 6， EIse continue

Step 4. Infeasibility test:

If i such that Yi < 0 and

Yi -L min(O， aij) < 0

jGT

Go to Step 6， EIse continue

Step 5. BaIas branching test: For each free variable巧createthe set叫

叫=(i: Yj -aij < 0)

If aII sets Mj are empty Go to Step 6. Otherwise CaIculate for回仙台ee

variable巧

巧=ヱ仇-aij)

ieM}

where竹=0 ifthe set叫 isempty. Add to the current partiaI solution the V紅iableXj由atmaximizes 巧・Goto Step 2

Step 6. Modi勿 Uby setting the sign of the righ加 ostpositive component. AlI elements to the right are O. Retum to Step 2 If there are no positive elements in U，血eenumeration is complete. The

・ ;__.dI

optimaI solution is in the incumbent vector X-. The objective is in.r. Jf.r is伺，

there is no feasible solution.

Balas.m is出em-file 白紙 implementsthe BaIas algorithm. It was translated from a PascaI program available in Reference 6. It is a program and prompts the user for the problem information-n， m，A， b， c.百letest problem given in Reference 6 is included below.

Exampk8.5

Minimize f(X): lOx. + 14x2 + 2lx3 +42x4

A l~ ~~

8.3 AOOlTlONAL EXAMPLES 347 町長一山一ケドTi--vMW/信雄戸P

Subjectto: -8x 1 -llX2 -9X3 -18x4 S -12

-2x.-2x2一7x3-14x4S-14

-9x. -6x2 -3X3 -6x4 s-lO

X.. X2' X3. X4 E [0 1) 〈バMmrd司li川川町5UH品h

Theoutput 合omBalas.m:

Solution exists ~

ーーー・・ーーーー---ーー----

The variab1es are: 1 0 0 1

The objective function 52.00 constraints -Va1ue and Right hand side

-26 -12 -16 -14 -15 -10

This corresponds to the solution identified in Reference 6.

Complete Enumeratlon: The Balas aIgorithm performs p~a~ enumeration. The complete or exhaustive enumeration (n.) for n binary variables is given by the relation

ne=2n

Typically，erIteEprise decision making involves several hundred variables and linear models. On the 0出erh叩 d，engin回 ringproblems characterized by a nonlinear model may have about 20 binary variables mquiring ne=l，O48，576enumBEaIions-This is-no bi'deal with desktop resources today.KEep in mind白紙 compl巴teenumeratlon

avoids any relaxation solution (purely discrete problems only).

3i-ijii

CaIlSimplex.m:明白 isa function m-file that c叩 beused to solve LP programs. It cm beusdtoobtain solutions to LP mlaxation models during the BB SEarch.It is based on the aIgorithm available in Reference 6. It is translated from a Pascal program.

百lemathematicaI model is

Minimize cT x

Subject to: [A]匹]= [b]

x~o

Slack variables are incJuded in出ecoefficient matrix [A]. Greater出制 orequal to constr凶ntsc佃 beset up with negative slack variables as the program will


automatically execute a dual-phase simplex method. Another altemative is to出巴白e

LP programs provided by MATLAB. The function stalement for using Call

Slmplex.m is

function [xsol， f] CallSimplex(A，B，C，M，N)

To use the function. A. B. C. M. N have to be defined. M is血巴 numbcrof constraints and N is the number of design v釘 iables.

REFERENCES

1. Fletcher. R.， Praclical Melhods olOp'imizalion， Wiley， New York， 1987.

2. Papa加nbros，P. Y.， and Wilde， J. D.， PゆlCiples01 Opli~聞I Design-Modeling and ConψUlalion. Cambridge University Press， New York， 1988.

3. Rardin， R. L.， Oplimilalion in Operalions ResearclJ， Prcnlice-Hall， Englewood CI江fs，NJ，1998.

4. Bellman， R.， Dynamic Progranun加g，Princeton UniversiLy Press， Princeton， NJ， 1957.

5. White， D. J. Dynamic Programming， Holden-Day， San Francisco， 1969.

6. Syslo， M. M.， Deo， N.， and Kowalik， J. S.， Dlscrele Oplimization Algorllhms，町宮ntice-Hall，Englewood Cliffs， NJ. 1983.

7. Ryerson Slock Sleel Lisl. Joseph T. Ryerson， Inc.， 1995.

PROBLEMS

8.1 Translate Example 1.1 into a discrete optimization problem and solve by exhaustive enumeration.

8.2 Translate Example 1.2泊10a discrete optimization problem and solve by exhaustive enumeration.

8.3 Translate Example 1.3 into a discrete optirnization problem and solve by exhaustive enumeration.

8.4 Implement a multistage exhaustive enumeration for thc continuous optimi却 tionof Example 1.1.

8.5 Transla飽 Example1.1 into a discrete optirnization problem and solve by branch

叩 dbound method.

8.6 Translate Example 1.2 into a discrete optimi四 tionproblem and solve by branch

and bound method.

8.7 Translate Example 1.3 into a discrete optimization problem and solve by branch

and bound method.

8.8 Identify appropriate st∞k availability for the flagpole problem in Chapter 4 and

solve the corresponding discrete optimization problem.

8.9 Solve Example 1.3 using山巴 stockinformation available in Ex8_ 4.m.

PROBLEMS 349

8.10 You are the owner of an automobile dealership for Brand X cars. Set up a DP

for monthly inventory control based on cars expected 10 be sold. high ordering

costs. unit costs for the car. and inventory costs for excess cars. Solve it using

the code in Ex8_3.m.

8.11 Solve thc following using Balas algorithm-using completc cnumeration

Minirnize f(X) = XI + X2 + X3

Subject ω3xI + 4X2 + 5x3ミ8

2x1 + 5x2 + 3X3 ~ 3

X¥. X2. X3ε[0 1]

8.12 Progr盲目1an exhaustive enumeration techniquc for zero-one IP using MATLAB. Solve Problem 8.10.

8.13 Is it possible to solve the capital budgcting ex創nple?If so， what is the solution

using complete enumeration?

8.14 Solve capital budgeting example using Balas program.

8.15 Create an example which involves making binary choiccs regarding your participation in course and extracurricular activitics 10 maximize your grades.

Solve it.

9

GLOBAL OPTIMIZATION

This chapter introduces two of the most popular techniques for finding the best value for the 0均ectivefunction， also referred to as the gIobal optimum solution. They are Simulated Annealing (SA) and the Genetic Algorithm (GA). The names are not accidenta1 since the ideas behind them are derived合omnatural processes. SA is based on cooling of metals to ob組indefined crysta1line structures based on minimum potentia1 energy. GA is based on the combination and recombination of白egenes in the biologica1 system leading to improved DNA sequences or species selection. In actual application， they ar官 morelike a continuation of Chapter 8 (Discrete Optimization)， because these methods can be used for continuous and discrete categories of the optimization problems. They share an important characteristic with other optimization techniques in白紙出eyare prim釘ilysearch techniques-they identify白eoptimum by searching the design space for the solution. Unlike出egradient-based methods， these techniques are largely heuristic， genera11y involve large 細 ountsof comput姐on，and involve some statistical rea50ning. Globa1 optimization is being pursued seriously today at回 ctinga significant amount of new research to白earea of applied optimization after a long hiatus. A significant drawback to many of thesetechniquesis白鉱山eyrequire empirical tuning based on the class of problems being investigated. There is also no 回 sy way to detecmine techniquel problem-sensitive parヨmetersto implement岨 automaticoptimization technique.

The global optimum cannot reall y be characterizedωbeing different from the local optimum. The latter is referenced when only a part of the design space (as op抑 sedto 出ecomple総 space)is viewed in the neighborhood of a particular design point. The standard Kuhn-Tucker conditions， established and used in earlier chapters (for continuous problems)， will identify出eloca1 optimum only. This is a1so more emphatic in numerica1 investigations of continuous problems where the algorithms are expected to move toward the op“mum values close to where th巴:yare started (initial

350

9.1 PAOBLEM DEFINITION 351

~uess)， While it makes sense to look for the globa1 optimum，也巴飽 i包5no method to

白LEn6ιffv 伽 Jchar叫帥 u叫凶lt t c側 inふuo1u凶1路sc.ω伽'01仰m附v.附Fぽe町x;Emいroωo油blems蹴 g伊u凶aran附帥附a刊岬E以l伽州Is仙oω仙山lu蜘 I仙ti山is削 d雌i江iffi伍fic伽刷帥cω叫州

tωoa釘閉剛E卸t路削t由伽ha“tSl帥 ad尚蹴re叩pres鈴en附tωson均lyami加nis印叫削cα凶u山』

r叫 :みP帥削i捌 o叩np仰ωb凶lem附s紅御巴日n削otn附巴郎ces郎s叩州lyc∞伽o叩n附v巴此 Therefore， i加nt蜘h恥el蜘it加t匂臨e釘r酬眠

both oft血he鈴semethods are exp巴ct飽，ed~ωop戸roduce solutions c10se ~ωot白h巴 global optimum solution， rather the globa1 solution血emselves-'I11isis notto suggestthat investigation oftheglob山 oluuOnisa wa蜘 of幽 eif the possibility of one叩notb巴倒油lished

;立かお古222221;ti::;fr;;LP222ぷJZ;210ptiIIlizaGon techniques are simple to implement while mquutng exhaustiVE calculations.This can potenudly soak up all of山eidle tim巴onthe persona1 desktop

computing resource， le叫 ngit回mediatea町activeness.23e techniques and their ascussIon axe VEry basIC following血E白emeespoused

in 円 apter8. 百e問 deris encouraged to 由 it journal publications ω monitor 血臨巴

ma酬aU帥1IよEzzEodf 血伽帥白悶巴慨帥帥h加ni榊q中仰脚u田附巴鈴s.ln 白刷帥i泊sc伽h加apt飽e肌白耐伽伽r悶附sはts叩o佃叩m鵬ei…s騎捌叫叩s叩削sues凶附巴郎sr時e曙例判g阿凶凶副叫i加均n昭gg脚10凶

刑同阻uω帥o叩n紅蹴'ep附 n】t叫 f刷011伽1


τ'h巴followingdcvclopmentor-globaloptimizadon is bas don unconstrainedcontinuous op自国zation.Solving cons回 inEdpmblemsdGemotappeartohaveevolvedtoacommon tec加向田血dis not illus回 .tedin this book. Note也atcons回 inedproblems can be converted to an uncons佐官inedproblem using a penalty formulation (Iαlap町 7)・明le

problem can be defined as

Minimize f[X); [X)n

Subjectto: x! 三 Xj S. X~

(9.1)

(9.2)

百刷sproblem were削抑制 (onlyone捌加um)，白eop削捌oncould 切手lybe 回 tablishedby any of the a卯ifOpriat巴techniquesused in the b∞k since the loca1 minimum is also也，eglobal minimum. To也，vestiga胞 theglobal min田町nit is therefore expected 伽巾帥附uldbem叩 ylo仙 ninirr剛 15，several of伽 nhavingd臨 rentva1附おr批objective.宮崎leastamong them is then出eglobal solution. Loca1 minimums， in血eωseof continuous problems， are血ose白紙sa出fy血eKT conditions，佃dwill be numerically determi耐 jf白estarting仰 sw俗 in血enei酔bo出oodof胞 p副叫minimum.This definition may be applied to discre島eop自国zationproblems that us巴continuousrelaxation ωob阻in曲esoMUon-For discmte problems using comple飽 enumeration，the global ootimimuon Should hdiscoverEd automatically.FIT1eEEfGEe-the discussion applies to

discretep帥 lemsthatu鴎 onlyincomplete enumeration.

9.1.1 Global Minimum

The global rninimum for the functio~f(X) exists ifぬefunction is continuous on a

nonempty feasible set S出atis closed and bounded [1]・

352 GLOBAL opnMIZATION

百leset S defines a region of出edesign space白紙 isfeasible. Any point in S is a

candidate design point X. In the n-dimensional Cartesian spa田， the point X

co町田pondsto a vector of lcngth IIXII. The boundaries of the region are part of the s巴t

S (c1ose，の.官lelength of the v民 torX should be a finite number (bounded).百1巴

definition for the global minimum X. can be statedω

f(X.)壬f(X); Xe S (9.3)

The conditions for the existence of the global minimum as given above are attributed

to Weierstrass. While白eglobal minimum is guar叩 t田 dif the conditions are satisficd，出eydo not negate出eexistence of the global solution if conditions are not satisfied.

Considering the broad range佃 dtype of optimization problems， the conditions ar芭fairly limited in白血use釦lness.The conditions for血巴i田 alm泊回umare白esame

as constraint (9.3) except the set S is limited to a small region around X..

Convexlty: If an optimization problem can be shown ωbe convex， then山elocal

minimum will also be a global minimum. A convex optimization problem requir，ωthc

objcctive function ωbe convex. If constraints are present， then al1 of由巳 inequality

constraints must be convex， while the叫ualityconstraints must be linear.

ConvexSet: 百lIsusually relates to由巳 designvector (or design points) of出e

problem. Consider two design points X. and X2・Asbefore， the set S defines a design

region and by implication bo白 X.and X2 belong to S. The line joining X. and X2

contains a whole series of new design points. If al1 of these points also bclong to S，山enS is a convex set. A cIosed set S rl叫ui陀sthat the points X1姐 dX2 can also be on

the boundary. If， however. the line joining X. and X2 contains some points that do not

belongtoS.出enS is not a convex set. For a two-dimensional design problem. convex

叩 dnonconvex sets arc illustrated in Figure 9.1.

Convex Funct/ons: A convex function is defined only for a convex set. Once

again let S be a convex set from which two design points X. and X2 arc selccted.

Letfbe the function whose convexity is to be cstablished. Consider any point X"

on a line joining X.佃 dX2. The functionfis convex ifj{XJ is less白畑山 va1ueof白ecorresponding point on白巴linejoiningftX.)andftXu. ln n dimensions山isis

not easy to se巴buta one-dimensional representation is prωented in Figure 9.2.百le

condition for convexity in山efigure is

f(x，，)Sr (9.4)

A mathematical condition of convexity is that th巴Hessianma凶x(matrix of second

derivativcs) of the functionfmusl bc positive semidcfinitc at al1 points in the set S. Note thal this corresponds to出eKT conditions for unconstrained minimizution

守~tすで!!.":~寝間=<ヨ

一「寸d


安

s NonconvexSe.

X2J

I X1 ~十f時

Xl

円gure9.1 Convex and nonconvex se!.

34支イリ担問円山山仰いkuohy山山戸一言明弓iJ

problems. From a practical viewpoint. the eigenvalues of自己 functionmust be greater

山anor equal to zero at al1 points in S. lt is important to realize白紙aglobal optimum may sti1l exist even if the convexily

conditions are not met. ln real optimization problems. it may be difficult 10 estublish

convexity. In practice. convexity checks are rarely performed. An assumption is made 白紙 aglobal optimum exists for the problem.明白 isa subjective element白血forced

by experience. insight. and intuition. This is also often justified by白epossibility白紙

a better solution than山ecurrent one can be found.

x X] Xa X2

Flgure 9.2 Convex functlロn.

354 GLOBAL opnMIZAnON

9.1.2 Nature of the Solutlon

It is assumed出創出eglobal optimum solution wiIl be obtained numeri心a11y.A good understanding of the nature of the solution is possible with referenc巴ωacontinuo叫design example. Exωnsion to di鈎re胞 problemsis possible by considering that any continuous solution co即'espondsto a specific combination of the set of disc陀 tevalues for the design variables. Figure 9.3 is an i1Ius町出。nof a figure with scveral minima and maxima with a s位ongglobal minimum at x， = 4， X2 = 4.百lefig町e，available 血roughflg9_3.m，1 is the plot of白efunction(aIso Example 9.1)

子、 sin(0.1 +ぜ(X.-4)l+(X2-4)勺 (9.5)f(x.・Xz)=-20

.. --~， 0.1 +ぜか1-4)z+(x2-4)Z

Figure 9.4 iIlustrates白econtour plot The contour plot can be bet旬rapp児ciatedin colorusing出直colorbarfor interpretation. Running the m-fiIe will create both figures. From relation (9.5) it is cIear the ripples wiIl subsidize as XI and x2 move much farthcr away from where the global minimum occurs， be氾aus巴 ofth陪 influcnceof the denominator. The numerator is constraincd to Iic between -1 and + 1 multiplied by thc

Several mlnlm目白ndmaxlma wlth global solullon

5

。

5

0

qH£一

-15

-20 10 .t::

x z

-10 -10

Figure 9.3 Surface plot of Exampl由9.1.

Ipiles 10 be downloaded 針。mthe web si!e ar芭indicnllldby boldface snns serif typc.

づ民

事

9，1 PROBLEM OEFIN町ION 355

理

Several mlnima and maxlma with global solution-contour plot

14九日丈

1522F5Jl

~ ~ ~ ~ 024 6 x， values

Flgure 9.4 Conlour plot of Example 9.1.

scaling factor of 20. In the limit the ripples should start to become a plane. This can be confirmcd by executing flg9_3.m with different ranges for山 variables.

From Figure 9.3 it is easy to spot白巴 globaIsolution by observation. Surrounding 出eglobaI minimum are rings of 10四 1extrちma.Numerically it is a chaIlengc 10 negotiate a1l of the bumps and dips. A simple way to charactc白.enumerical techniQues for a minimization problem is that白eyattempt to find a ditch (or valley) to faIl into. If this is胸凶asan unconstraincd minimum problem， the K1・conditionswilllead the solution to be回 ppcd泊四yoneof也elocaI minimum rings， depending on the location of出cstarting point for thc iterations. lf the iterations創'Cstarted cIose to X( = 4， x2 = 4，出enit is more白叩Iikely出cglobal optimum wil1 be discovered. Staring atxl = -5， X2 = -5， using standard numerical t田 hniqueswill probably not lead

to出巴 globaIminimum. This is a simple illustration but notGthem areManitBlocal minimums and

m蹴加ums制 daEingleglobalmmm.In 則的ignproblems wi白morc白叩2V紅 iables，it is not readily apparent how many locaI minimums exist， and if出eglobaIminimum is rcally that distinct， even if it is postulat吋 tocxist. What about the complcte enumeration technique? Thcre is no way thc globaI minimum can escape thc

356 GLOBAL OPT1MIZATION

sifting due to血istechnique. There is a1so no technique山atis simpler出佃completeenumeration.百leproblem is Ihe time to finish searching Ihe infinite points of Ihe design space to discover Ihe solution.

The solution to global optimum must Iherefore lie between standard numerical techniques and complete enumeration. The former will not find Ihe solution unless it is fortunate. The 1atter carmot find Ihe solution in time to be useful. Global op也凶zation飽chniquestend to favor the latter approach because of出esimplicity of imp1ementation. To be凶 efullhesolution must be discovered in a reasonable time.

9.1.3 Elements of a Numerlcal Technlque

百leprob1ems faced by a numerical technique searching for Ihe global minimum have b田 ntouched on in血eprevious section. In most standard numerical techniques白E

loca1 rninimum serves as a strong ma伊 etor a beacon for Ihe solution. In most cases， design changes are not accepted if Ihere is no improvement in the solution. If standard numcrical techniques have to be adapted to look for global rninimum， Ihen Ihey must encourage solutions出atin曲eshort run may increase Ihe objective value so白創出eycan find anolher valley at whose bottom is山eglobal minimum. They must achieve Ihis wilhout sacrificing由巳signific叩 tpotency of Ihe local rninimum. because a local minimum solution is better than no solution at a11. Figure 9.5 illustrates Ihese ideas using a single variable.

In Figu民 9.5出eiterations sta氏 atpoint A. In a standard n山neri回 1technique. a S田氏hdir田 tionis found which leads to point B. At B. if∞nventional numerical techniques were used， a search direction leading to point C will be accepted， while a search direction leading to point D will be strongly discouraged. But， if point D was somehow reachable，出enit cou1d move on to E and出enF and continue to Ihe g1oba1

f(x)

x

F1gure 9.6 Strategy for global optimum search.

主ぉ3 2

事

9.2 NUMEAICAL TECHNIQUES AND ADDITIONAL EXAMPLES 357

minimum. The main idea in finding the global minimum is to encourage solutions山atwill escape the al回 ctionof Ihe local minimum. This means al point E it is quite possible白紙白esearch direction may 1ead to point G. 11 is血enrl叫uired出ata 1arge number of iterations be permitted so白atIhe va11ey ho1ding Ihe g1oba1 minimum wi1l be revisited. From白isbrief i1lustration Ihe characteristics for a technique凶 edto find a global minimum can be summarized as:

• Short-term increases in the objective function must be encouraged.

• Traditional gradient-based techniques will not be particu1ar1y useful b巴cause出eyare based on search dir芭ctions曲atd民間槌eIhe objective function at Ihe current itcration. They wi11 not pcrmit incr回路 ofIhe objective function value.

・Ifdescent-based search directions are not part of Ihe technique，山叩 anydirection白紙decre酪 es出eobjective shouId be accepted to promote attraction by Ihe 10cal rninimum.

• As批 locationof Ihe global minimum is not known， search must be permitted over a 1argc region wilh no bias， and for a 1arge number of iterations.

• Since acceptable directions can decrease or increase山 obj削 vefunction value， a hcuristic determination of山 S同町hdirection (or design change) must bep訂tofthe旬chnique.

• These features resemb1e Ihe zero-order technique for unconstrained op白瓜zation叩 dIhercfore will rcquire a large number of iterations to be effective.百lelarge number of iterations wi1l force Ihe technique 10 be simp1e.

・Thesolution can only be concluded based on Ihe number of iterations， as necessary conditions (availab1e for continuous problems only) cannot distinguish between loca1 and global optimum.

Wilhout a doubt， Ihe best technique is complete or exbaustive enumeration. It is Iherefore necessary to assume that complete enumeration is IlOt all optioll in further discussion.

The prob1em description， illus岡山n，anddisc凶 sionhave dealt wilh uncons町ainedrninimization. Cons佐aintscan be included by addressing Ihe above characteristics to apply to feasible solutions only. It is 1eft to Ihe reader to cont巴mplate曲eability to he凶 stica1lygenerate a large number of feasible design changes for consideration.

9.2 NUMERICAL TECHNIQUES AND ADDITIONAL EXAMPLES

This section introduces two popular melhods for global optirnization. Bolh are derived from naturaI physical process自由athelp identify optimum states.百lefirst one， Simulated Annealing (SA). refers ωa physical process曲atestablishes differcnt crysta1line structures， each of which can be associated wi出 differentminimum potential energy. This is attempted by reheating and cooling of白巴 materials/meta1swhile controlling由巳raleof cooling.百lesecond， Ihe Genetic Algorilhm (GA)， works through mutalion of candidate solutions untillhe best solution can be eslablished， a

358 GLOBAL OPTIMlZATION

process曲atcan be related to natur百1solution or biological evolution.百lesetechniques are iIIustrated with respect to unconstrained problems in this section.

9.2.1 Slmulated AnneaJlng (SA)

Thc first application using SA for optimization was presentcd by Kirkpatrick et al. [21.The examples covered problems from electronic systems，in puticular， component placement命的enby the need for minimum wiring connections. The algorithm itself was based on the use of statistical mechanics， demonstratcd by Metropolis et al. [3]， to establish thermal cquilibrium in a col1cction of atoms. The term itself is more reprcsentative of the process of cooling matcrials， particularly metals， after raising their temperature to achieve a definitc crystalline statc. As mentioned earlier， the extension to optimization is mainly hcuristic. Thc simple idea behind SA can bEillustrated through the Inodiaed basic algorithm of SA due to Bochavesky ct al. (4).

Algorlthm: Baslc Slmulated Anneallng (A9.1)

Stcp 1. Choosc starting design "0. Calculate fo = J("o)

(Necd stopping critcria)

Step 2. Choos巴arandom point on thc surface of a unit n-dIlIII:Rsional hypcrsphere (it is a sphere in n-dimensions) to establish a search direction S

Step 3. Using a stepsizeα， calculate

ft =f(Xo +ω); Af=ん一λ

Step 4. If AfS 0， then p = 1

Else p = e-fJt.f

Step 5. A random number r (0 S r < 1) is generated.

If r S p， then the step is accepted and the design vector is up由ted

回se，no ch佃 geis made to出edesign

Go to Step2

If implemented，出巴 algorithmhas to executed for a reasonably large number of iterations for the solution to命的toa global minimum. The simplicity of the algorithm is very appealing. A ca蹴r陀efi“削110伽bsc仰rva叫6ωo叩nwill i加nd此i比ca蹴t旬e 白伽a剖tα 副d~ 蹴 i加m卯巾n瓜t pa創ra混加a

t白hep町叩1削ete釘r~ca叩n berl叫ela飢t吋 tωot山heBoltzma聞nnp戸ro曲ba油.bi出litydiおst凶rib加utωiぬo叩n.In s“ta凶tisticalmechanics it is exp陀 ssedas -klT; T is considered出cannealing temperatu問， andk出eBoltzmann constant. It dictates the conditional probab出守山ata worse solution can beac四 pted.For the algorithm to be effective， it is陀commendedthat p be in the range 0.5 S p S 0.9. In formal SA terminology， ~ repr芭sents出eann叫 ingtemperature. The foIlowing ideas in the algorithm arc apparent:

£一司詔V

9.2 NUMERICAL TECHNIQUES ANO AOOtnONAL EXAMPLES 359

• The algori血mis heuristic.

• The algorithm is suggestivc of a biased random walk.

• Descent of the function is directly accommodaled.

• Dirl回 tions出atincrease出evalue of the function are sometimωpermitted. This is important to escape local optimum. This is the potential to discover th巴globaloptimum through出ealgorithm.

This basic algorithm can defmitely be improved. During the initial iterations it is necessary to encourage solutions to escape出巴localoptimum. After a reasonably large number of itcrations， the opportunity to cscapc local optimum (window of opportunity) must be continuously rcduced so that thc solutioll docs not wandcr all over the place， and canccl the improvements to the solution出athave already taken p1ace. This can be imp1emen凶 byscheduling changes泊白ep鉱ameter~ぉ肱iterations increase. In SA tenninology，出isis referred toぉ白eannealing schedule. 百lescmodifications do not in any way decrease the problem!user dependence of the algorithm. This problem dependenc巴isusually unattractive for automatic or standard implementation of the SA mcthod. Rcsearchers are engagcd in construction of algorithms山atdo not depend on uscr-specified algorithm contro1 paramete路 [5J.

The SA algorithm for continuous variables implcmented in the book includes many of自己 basicideas from the original algorithm. There are many additional changes no凶 he陀.First恥 negativesign in the prob油ility旬rmis folded into ~ itsclf. A si町 leprocedure is included for obtaining the value of~. It is based on the initial valuc of "0 and fo. ~ is ca1cula凶 ωyiclda value of p = 0.7 for a Af叫ualto 50% of恥

ini叫ん百ealgorithm favors a negative value of ~ by d凶 i伊.Sincc the process for establishing an initial ~ i1Ius回 tedabove can detcrminc a po副 vevalue， the algorithm corrects this occurrencc. It is latcr illustrated through numerical experiments that a smal1 magnitude of thc pararncter ~ allows the solutionωwander all over the p1acc， while a 1arge value provides a s釘'Ong10cal at位action.Instead of the SA step being incorporated when出巴objectiveincreases， it is triggered when出巴objectiveis 1ikely to increasc bascd on胸 slope.In白ea1gorithm below， the co口紅01parameter ~ is kcpt constant at its initial value. The calculation of αis borrowed from the traditional one-dimensional stepsize calculation using the Golden Section method (Chapter 5) if the objective is like1y to decreぉe.If the slope of thc objective is positive or zero， a fixed αis chosen. This value is a maximum of one of the previous

Algorlthm: Simulated Anneallng伺9.勾Step 1. Choose "0.ω叫蹴J("o)，p = 0.7， ~ = -ln(p)/0.5 xJ("o)

百刷 <l，s= 1， If s> O，s=ーlxs

360 GLOBAL OPTIMIZATION

Step 2. Choose a random point on the surface of a unit n-dimensional hypersphe陀

to establish a search direction S

Step 3. Calculate Vj{Xo);

If Vj{Xo) < 0， then compute I・Dα.

AX=α・S; 文=Xo+AX; Xo← 笠 ←i+ 1

If Vj{Xo) ~ 0，

4f=f(為+尚一f(丸); a=m出 (α・， 1)

p = e~1lf

A random number r; (0 S r < 1) is gencrated.

If rSp，白en山estep is accep包dand出edesign vector is updated

AX=aS; 文=Xo+d.X; 為←文 i←i+ 1

Else continue i←i + 1

00 to Step2

The algorithm is availablc in SimulatedAnneallng.m. It is called through Call_SA.m. The algorithm is applied to Example 9.1， which was used for iIlustration in Scction 9.1. To executc the algorithm the following m-files (introduced in Chapter 6) are ncccssary: UpperBound_nVar， GoldSection_nVar.m. Following the practice of drawing contours for tW(トvariableproblcrns， two files are necessary for describing出.eexample， the second prim釘ilyto quickly draw a contour plot using ma佐ix calculations. In this case the files are Example9_1.m and

Example9_1 plot.m. The latter file name must be set in SimulatedAnnealing.m. The calling statement and variables are cxplained in SlmulatedAnneallng.m. Th巴ycan be read by typing

>> help SimulatedAnnealing

in the command window (the current directory must contain the files) The隠 aderis strongly encouraged to run the example 00 his computer. It would be

a singular coincidence if any of the 問sultspresented here are duplicated. Running numerical experiments is simply a mattcr of calIing the SA algorithm and taking a look at the rωults (especially出eplot). The design changes are illustrated using colors: red-represen出 gd民間asingslopes-and blue-for positive slopes伽 tlead to

design changes. The巴n回 pmentin the local minimum and the es回 pear芭c1earlyiIlustrated on the contour plots. The contour plots紅 efilled color plots to identify areas

on the figure where the function has a local minimum or maximum. Darker colors 陀 presentminimums and lighter colors the maximum. The examples were run usually

for 1000 iterations. It is conceivable白紙 theycould have becn stopped earlier. On a

200・HzPII laptop it required less由阻 aminute for each run. A large number of

iterations is useful to indicate if the solution escapes the global minimum once having obtained it. While it was not cvidcnt in出iscx創nple，it is likely problem dcpendenl since in山iscxample Ihe global minimum was significantly larger山anthc local

J7P912

•

9.2 NUMERICAL TECHNIQUES AND ADDITIONAL EXAMPLES 361

ι " 宮ぎ

minimurns. Several features of the SA algorithm are iIlustrated using some numerical

expenments.

Example 9.1 This is the function used in Seclion 9.1. The global minimum is at X・=[44] with an 0附田tivevaluej" = -19.6683. In出efigur民 thecontours suggest a

strong global minimum encircled by altemate local maximums and minimums. For

most of the runs血estarling value was far from the global minimum.

、 sin(0.1 +吋(XI-4):L + (-'1. -4)勺f(x.， x2) = -20

l' ~2' - -~ 0.1 +ぜ(x1-4):L + (x

2 -4)l

(9.5) Minimize -

pd

話

1一3Fフ-32qp、も毛布H

K

Z

一525ι12222

Five numerical experiments are presented below. It should be emph鎚 izedthat由ese

runs cannot be duplicated. Overall， the algorithm worked very well. For exactly出巴

same starting values and number of iterations (calling Call_SA.m without any

changes-about40也市s)出eglobal minimum was found 95% of the time.

Run 1: Starting design = [-5， -5]; s =ー2.7567; Number of iterations = 1000

Useful number of itcrations (when dcsign changes occurred) = 383

Final design = [3.9992， 3.9997]; Final Objective =ー19.6683

Figure 9.6 i1lustrates the trace of the design chang田.It escapes the starting relative minimum fairly easily to get 回 .ppedby a different local minimum. It is able to escape to the global minimum where it is strongly attrac包dto it.τbe位置.cesin the local

minimum appear to prefer to stay in the local minimum suggesting the solutions are

not being encouraged 10 escape.

Run 2: Starting design = [4，4]; トー0.0362; Number of iterations = 1000

Useful number of iterations (when design chang巴soccurred) = 991

Final design = [100.72， 127.75]; Final Objective = 0.128

Figure 9.7 iIIu柑 atesthc trace of山esearch for the global minimum. Most of the points are not in the plot as can be seen from the solution. This solution was started at the global minimum. The search process app回 rsto wander all over the place and finally off山巴 map.There is no pattem in th巴limitedpoints secn on the figure， except血esuggestion曲atthe search is encouraged toωcape. This can be ticd to the low

magnitude of s.

Run 3: Starting design = [4，4]; s = -1; Number of iterations = 10∞ Useful number of iterations (when design changes occurred) = 553

Final design = [3.8832， 4.3709]; Final Objective =ー19.173

Figure 9.8 ilIustra陥 thetrace of the design search. In this run the value of s was sel to -1. Once again the starting value of the design was the global minimum itself. From

the fig町eit is c1ear th鉱山esearch is not allowed to escape出eglobal minimum that

easily. The choice of the control parameter was arbi佐町y，and several ex創npleshave to be studied 10 see if山isvalue is problem independent. The value of I is typically a

363 9.2 NUMERICAL TECHNIQUES AND ADOmONAL EXAMPLES

明

ω2-倒、，剖UA

I手/

5

GLOBAL OPTIMIZA T10N

10

a

B

4

2

自

u

gagNM

362

o 2 咽 lu鈎

-2 x

' -4 -6 -8 10 B s 4 o 2

values -2

X1

4 ー自-6

Simulated annealing斗 owp.

Discussion on the Numerical Experiments:τ'he following observations are based on the experience of applying SA (A9.2) to Example 9.1. They correspond to 出ereported experience with SA in山egeneral literatu陀.

F1gure9.7 Slmulated anneaJing Example 9.1ー-convergedsolution.

good choice for optimization a1gori出ms，p紅白ularlyfor a weU-sca1ed problem. What is clear from the runs so far is that a negative va1ue of s of the order of 1 implements a loca1 at凶 ctor.

Flgure9.6

• Annealing schedule (changing s) is叩 importantconsideration in SA. Starting with a low va1ue出econtrol p制 me胞rs must be increascd-not too fast or it will get回 pped泊 aloca1 minimum.

• Annealing schedul巴 isdeveloped through tria1 and error， insight白紙 isnot obvious， and based on probable behavior of the s田氏h.

• Length of time (or number of iterations) that出eproblem needs to be searchcd is difficult to d巴termine.While山iscan be based on monitoring， given the stochastic naturc of the search， it cannot be asserted that出eglobal minimum has been estab1ished. The SA algorithm itself can be expected to converge while this property is too slow for practical use [6]. A larger number of iterations is more likely to yield the globa1 solution.

• SA search does not subscribe to iterative improvement. It i5 an example of a stochastic evolutionary process. The implementation uscd here陀P問seotsa memoryless se鉱山.

Run 4: Starting design =ト5，-5]; s = -2.7567; Number of iterations = 1000 Useful number of iteratioos (when design changes occurred) = 558 Final design = [-3.3397， 6.2019]; Fina1 Objective = -2.5674

This is a repeat of Run 1: Fi肝民9.9traces the search for the globa1 minimum for this run. It appears白at1000 iterations is not sufficieot to determioe the global minimum. While the solution has escaped the starting loca1 minimum， it has not yet escaped出enext local minimum. Since the va11eys (loca1 minimums) in this example are circular， note出atthe solution moves around in the va11ey but has not managed toωcape.

Run 5: Starting design = [-5， -5]; s = -2.7567; Number of iterations = 10，000

Useful number of iterations (when design changes occurred) = 3813 Final design = [4.0037， 3.9996]; Final Objectiv巴=ー19.6682

No figure is presented. The solution converges to the global minimum.

" 3 t

365 9.2 NUMERICAL TECHNIQUES AND ADDITIONAL EXAMPLES

gaE'NH

言

一伊2404芳

bp者一旬ιqdNJS詰tAqdfd

GLOBAL OPTlMIZATION 364

gaE'同

M

10 B e 4 -2 0 2 η-val凶 S

-4 -6 -8 10 8 自4 -2 0 2 X， -values

-4 -8 -8

Dlscrete Problems: In discrcte problems， the SA algorithm will use a discrete

design vector det巴四unedstochastically合omthe avai1able set Each element of出e

design vector is randomly chosen from血epermissible set of values. The remaining

elements of th巴SAalgorithm are directly applicable. If山eobjec凶vedecrcωcs， then the design change is accepted. lf it increascs， then a conditional probability of

acccptance is determined allowing the solution to escape thc minimum. This is lcft to

出巴隠ader.The significant effort is the identification of山erandom design vector.

Example 9.2 This is a function of two variables that MATLAB uses for iIlustrating the graphics capability of也巴software.Typing peaks at the Command prompt will draw

the surface plot. Typing ηpe peaks.m at the Command prompt will provide血e

叫uationsthat were uscd to obtain the curve. This function used as山eobjectivc is

available in Example9_2.m and Example9_2plot.m. SA is used ~o identify the global minimum. Figure 9.10 illustrates the contour plot a10ng wi自由巴changesin出e

design. The function has two local minimums and thr田 localmaximums. The

following reprcsents a converged run.

Slmulaled annealing: non屯onvergence-lnsulficienlllerallons.Flgure9.9 Slmulaled anneallng: Increaslng β-decreas自doppロばunityto escape. Flgure9.8

• Rest釘 tingfrom differcnt d巴signmust stiU be implemented to overcome strong local influences.

• Restarting from the same initial de討伊 isa1so important for establishing convcrgence prope凶esas出isis a stochastic search process.

Constrslned Problems: This chaptcr does not deal with constrainls or discrctc problcms.百lercarc two ways of handling constraints dircctly

1. lf the new design point is not feasible， it is not accepted and no dcsign change tak回 place.This is quite 1四回ctivcfor random s回民hdircctions.

2. 百lefunction出atdrivcs the a1gorithm during each iterntion is the one with the largcst violation. If the current point is feasible， then the objective function will drive白巴 search.This make sense ω 出eprocess is largely stochastic and is

. expected to wander around曲edesign space.

Starting 的 ign= [-2， 2];トー2.5;Numbcr of iterations = 1000

Useful number of iterations (when desigo .changcs occurrcd) = 338

Cons紅泊ncdproblcms c叩 beexpected to take a significantly Iarger numbcr of

S伺 rches.The問 iscurrently no pr'町吋ureto identify convergencc prope町 ofthcSAwith即 sp田 tto constrained optimization.

366 GLOBAL OPTlMIZATlON

-4 -3 -2 -1 0 2 3 4 5 X1

values

Flgure 9.10 Sfmulated annealing: Example 9.2.

Final design = [0.02279， -1.6259]; Final Objective = -6.5511 This is the global minimum of the problem.

There was a significant change made to出eSA algorithm based on initial旬stswi出血eex田 ution.At the starting design， the design was largely positive (8.95). Changing it to a negative number did not permit th巴solutionto escape the locol rninimum or to swing oround the nearby local maximum. Numerical experimenls suggest出鉱山econstant f3 be maintained between

一2.5sf3S-1 (9.6)

Otherwise no change was made to山ealgorithm.

9.2.2 Genetic Algorithm (GA)

百leGenetic Algorithm is an important part of 0 new area of applied research termed Evolutionary Computation. Thcse are actually search procωses and naturally useful

9.2 NUMERICAL TECHNIQUES AND AOOITIONAL EXAMPLES 367

for discovering optimum solutions. Holland [7] was the first to usc the technique， but iωuse as an optimization tool began in eamest in 白elatc 19808， developed momentum

in the mid-1990s， and continuωto attract serious interest to血y.The term evolutionary is suggestive of出enatural process associated wi出 biologicalevolution-primarily

the Oarwinian rul巴ofthe selection of the fittest. This takes place because出econstant mutation and recombination of the chromosomes in血epopulation yield a better gene

StructUI芭. A11 of these terms appe釘 inthe discussions related to evolutionary computation. This basic pro田 ssis also folded into白evarious numerical techniques.

The discussions can become clear if恥 termchromosomes can be associated with design variables. Over the y国民， Evolutionary Computation has included Genetic Programming-which develops prograrns白紙 rnirnick山eevo1utionary proc蹴;

Genetic Algorithmsー optirnizationof combinatoriaVdiscrete prob1ems; Evolutio，胤 ryProgramming-optimizing continuous functions without recombin甜on;

Evolutionary Strategies-optirnizing continuous functions with recombination [8]. This c1assification is quite loose. For example， GAs have been used， and continue to be used effectively to solve continuous optimization prob1ems. Evoh瓜onaryalgorithms have been successfully applied to optimization prob1cms in several areas:

enginecring design， pararneter fitting， knapsack problems， transportation prob1ems， image processing， traveling salesman， scheduling. and so on. As can be infe町吋 from山elist. a significant number are合omthc arca of discrctc programming.

This section and出isbook include only GAs， though al1 of the evolutionary algorithms share many common features. GAs generate and use stochastic

information in their implementation and therefore can be considered global optimization techniques. They are exceptional1y useful for handling ill behaved， discontinuous. and nondifferentiable problems. Unlike the SA， convergence to the globa1 optimum in GAs can only be weak1y established. GAs generate and use population of solutions-design vectors (X)-un1ike恥 SAwhere a search

direction (8) was generated. This distinction is essential to understand the difficulty of handling continuous problems in OAs. For example. using random

numbers to g巴neratea value for Xlo the values of 3.14，3.141， or 3.1415 will al1 be distinct and have no relation to each other.

Generlc Genetlc Algorlthm (GGA) (A9.3)

Step 1: Set up an initial population P(O)-an initia1 set of solution or chromosomes

Evaluate血 ini出1solution for fitness-di能 rentiate.collate， and rate solutions

Generation index I = 0

Step 2: Use genelic: operalors to generate the set of children (crossover， mutation)

368 GL08AL OPTIMIZATION

Add a new set of randomly generated population (immigrallts)

Reevaluate山 population-fitness

Pcrform competitive selection ・・・-whichmcmbers wil1 be part of next

generation

Selectpop凶ationP(t + l)-same number of members

If not converged t←t + 1

GoToStep2 The various terms are explained below.

Chromosomes: While it is appropriate to regard each chromosome as a desi伊

vector， there is四 issueof representation of the design vector for handling by genetic

op巴rators.It is possible to work with白巴 designvector directly (as in出eincluded

example) or use some kind of mapping， tl四 1(real encoding) or binary (binary encoding). Earlier work in GA used binary encoding. In出isbook the design vector

is used direct1y for a11 transactions. Real cncoding is陀commended.A piece of the

chromosome is called an allele.

Fitness and Se/ection: The fitness function can be related to thc objective

釦nction.τbepopulation can be rankedlsorted according to血eobjective function

value. A selection from this population is necess釘yfor (1) identifying parents and (2)

identi匂ingif出eywi11 be promo飽dto曲enext generation. There are various selection

schemes. A fraction of the best individua1s can be chosen for reproduction and

promotion， the remaining population being made up of new immigrants. This is ca11巴dtournament selection. A modification of出isprocedure is to assign a probability of

selection for白色 orderedset of the population for bo由reproductionand promotion.

This probability can be based on roulette wheel， elitist selection， linear or geometric rankillg [10]. The implementation in this book uses ranking without assigning a

probability of sel田 tion.1tuses the objective function value to rank solutions.

Inlt，加IPopulation: The paramet巴rhere is the number of initia1 design vectors

(chromosomes) for出estarting generation， Np'古山 numberis usually kept the same

in successive generations. Usually a norma1 random generation of the design vectors is recommended. In some instances， doping， adding some good solutions to白e

population， is performed. It is difficult to establish how many members must be

considered to represent the population in every generation.

Evaluation: The various design v配 torshave to be eva1uated to s田 if曲eycan be

used for crea出 gchildren白atcan become part of the next generation. Fitness

attributes can serv巴wellfor this pu叩ose.For the case of unconstrained optimum， the objective functionJ(X) is a good measure. Since由巳 globalminimum is unknown， a

progressive measure may be used.

副司星芸品l~û凶m;;ii;~品幽温

9.2 NUMERICAL TECHNIQUES AND ADDITIONAl EXAMPLES 369

Genetic Operators:百leseprovi也 waysof defining new populations from

existing ones.百lIsis also where the evolution takes place. Two types of operation are

recognized. The first is crossover or recombination， and the second mutation.

Crossover Operators: There are several types of crossover strategies. A simple crossover is ilI山tratedin Figure 9.11. Here， a piece of白echromosome-an

a11ele-is exchanged between two p紅 entsP" P2 to produce children C" C2• The leng血ofthe piece is usually determined randomly. Mathematically， if the parents訂e

X = [x" X2， . • . ，Xn] and Y (defined similarly)， and r is the randomly selected truncation index.白enthe children U (= C 1) and V (= C2)， by simple crossover are defined as [11]

x，九

九

あ

.

rt』百

11r14llL

-一一-

u

v

ifi <r

o出erwis巴

if i < r

otherwise

(9.7)

Arithmetic crossover is obtained by a Iinear combination of two parent chromosomes ωyield two children [9]. If Il.1 and 'z are randomly generated numbers， then the children can be definedぉ

y

y

h

h

+

+

xx

hL.

一一=qq If

(9.8)

いい1; 入i'ヤ0ーitis a convex∞I巾 ination

Pt

Parents

P2 Ct C2

Childrcn

円gure9


If there are no restrictions on λ's， it is an q伊necombination. For realλ's it is a linear combination. An important crossover operator is a direction-based crossover operator， ifJtY) is less thanJtX) and r is a unit random number:

c= r(Y -X) + Y

Mutation: Mutation refers to白ereplacement of a single element of a design vector by a randomly genemted value. The el巴mentis also randomly chosen. For曲巴designvector X， for a randomly selected element k

c=JXj， i*k IXk， i=k

There are oth巴rtypes of mutation including nonuniform mutation and directional mutation. The reader is referred to Reference 9 for a relevant description.

Imm.匂'fants:This is出eaddition of a set 01' l'andomly generated population in each generation before the selection is made for出enext generation. The search process in GA is broadly classified総出plorationand exploitation. Exploration implies searching across the entire design space. Exploitation is focusing more in the area promising a solution. Typically， crossover and mutation are ways to narrow the search to an area of design spa田 basedon血，echaracteristics of the current population-suggesting a local minimum. To keep由esearch open for the global mInImum， other areas must be presented for exploration.ηlis Is achIcved by bringing in an unbIased population and having them compete for promotion to血enext generation along wi白thec即時ntset. In many variations these are allowed to breed wIth曲ecurrent good solutions before evaluation.

The ideas presentcd here are basic to all of evolutionary computation印 dspecific to GA. They are also simple but complete. For each category listed above there are more sophisticated models白atare being researched and developed and出ereader is encouraged to seek information from the current literature. Thc specific GA algorithrn implemented in由巳 bookincorporates all of白eelements above in a simple implemen組組on.

Genetlc Algorlthm (A9.4) The algorithm actua11y implemented is given below. In line with many other algorithms avaIlable In thIs text， It Is provIded as a way to understand and appreciate the working of the GA. It is a1so meant to provide a starting point for the user to implement his own variation of the GA since these techniques are sensitive to血especific problem being s凶died.Another important feature mIssing 合om出Isimplementation is an encoding scheme. The design vector is handled directly to provIde a direct fcedback.

1."1':'、肺:W ~r~1:.，:c. 三... 昭三 JAよ4 ，1~，，'r.!.iIt";1:.;L'.:m~~

加を

RHU--

し

ιlJu--UFIFti--jin--Jr

9.2 NUMERICAL TECHNIQUES AND ADDITIONAL EXAMPLES 371

S旬P1. Set up an initial population. Ten sets of design vectors are randomly created. From this set， an Initia1 generation of two vectors is selected. This is 出巴 poolavailable for propagation. This is also the mInImum to perform crossovers. The fitness function Is出eobjectIve functIon.

Generation Index t = 1

Maximum genemtion Index = tmax

Step 2. Genetic Operations

Two Simple crossovers

Two Arithmetic crossovers (convex)

One Directional crossover

Mutation of all seven design vectors so far

Fourimr凶g細胞

Selection of the best two design vectors (from 18 vectors) for promotion to出enext generation

If 1= 1m 叫， Stop

Increment generation index: I←t + 1

Go to Step2

Example 9.3 The example for也issection is a Bezier curve fitting problem. Given a set of砂 data，find the fifth-order Bezier curve白atw迦白血eda也.百leobjective function is 血 leぉtsquar叫 errorbetw，田n出eoriginal data and the data generated by the Beziercurve. 百世sproblem has been adclressed earlier出rougha continuous deterrninistic technique in an earlier chapter.百ereareeなhtdesi伊 variablesass∞iated with the problem.百lereareseveral reasons for the choice of the problem. Fi隠しtheglobal solution， if it exists， must be close ωzero，胡d鉛 cond，白isprovIdes a way ωdescribe eight design variables in two-dim巴nsionalspace (plot).官lebest value for the design vector泊everygener甜onisplotted to see how the curve fit improves. Since encoding Is not being used in the algorithm， a reasonably long desIgn vector is necessaryωobserve th巴crossovers.百leoriginal data are obtained from the folIowing function:

Jtx) = 1 + O.25x + 2e-x cos 3x (9.9)

Figure 9.12 illustrates血eseωP of the example including the eight design variables， which are the x and y compo即時 ofthe intemal vertices伽 tdetermine曲efitted curve completely.τ'be function is hard coded in CaIlGA.m. There are several reasons why出isIs not由ebest example for illustration of GA. FIrst， it Is叩 examplein continuous variables. Second， local minimums are not very distinguishable and there are innumerable m朗 yof them. The global minimum is not substantially different from the local minimum value to w紅rantexplomtion of the globa1 solution. Third， the problem Is not ill behaved. The most important reason depends not on the algorithm buton出ecomputatIon of the objective function-ca1culatIng the least squared error.

372

2.5

2

g ~ 1.5

h

0.5

。。

GLOBAL OPTIMIZA TlON

02

、

x z

0.4

円gure9.12

0.6

Bezier curves

ーー Orfgln剖 Dala--Rlleddala o Vertlces

0 ・

(Xs'Xo)

。....…ト一一 /0・"い

旬、} ‘ 。ソ/.明日}

0.8 1 x-data

12 1.4 1.6 1.8

Genetlc Algorithm. description of Example 9.3.

2

It uses a Newton-Raphson technique出atis prone to fai1urc if thc curve folds over severely，or出cvertices are very c10se toge出er.One way to overcome this is to cause the x-Iocation of the v巴rticesto monotonically increase (which is implemcntcd in出epopulation gcncration function). This upsets many of the crossover operations SIOCC 出町arenowco叫伽allyrandom in蜘 dofp叩lyrandom.

百lefollowing files are nccessary for山ssp配 ificex創nplerelating to the handling of Bezicr curves: c僧符.m，Factorlal.m， Comblnatlon.m， Bez_SCLError.m and are available in the code dircctory for the chapter.

The GA in AIgorithrn (A9.4) is implemented using the following m-flIes.

CaIlGA.m GenetlcAlgorlthm.m populator.m SlmpleCrossover.m Ari肋metlcCros:関Iver.mDi開ctionalCrossover.mMutatlon.m DrawCurve.m

Setting up parameters and data for the a1gorithm Di陀ctsthe algorithm Gencrates random design vcctor Pcrforms simple crossover between two parents Performs ∞nvex crossovcr betw偲 ntwo parents Performs one dircctional crossover Performs mutation of出edesign vectors Draws thc curve for each design vcctor

可

9.2 NUME肉ICALTECHNIQUES AND ADOITIONAL EXAMPLES 373

Figures 9.13 to 9.17 capturc thc app1ication ofGA to Example 9.3 through 200 generations. Thc reader should be able to run the cxample from the code dircctory of Chapter 9. Because of the heuristic nature of the gcncration of solutions， the solution trail will be different from山atshown below. Running the examplc several times by invoking CallGA from the Command window should illustrate 白紙 theGA always manages to closcly approximate thc curve after 200 generations. The maximum gcneration va1ue is not predictable and cannot bc estab1ished with certainty. It relies on numerica1 experimentation. This number appears casily to be over 1000 in many investigations. The reader is urged to expcrimcnt with the example to cxplore this issue. In running CaJlGA.m the bcst values of the design vector and thc corresponding objcctive function valuc are capturcd in the variable Xbest-an array of 201 rows and 9 columns (assuming 200 generations). The first 8 eolumns carry the best design for the generation and the last column is the corresponding objective function value.

Fig町 e9.13 is白ebest design from the starting generation. It has叩 objectivefunction va1uc of 32.13.百lisis山ecπorsquared over 100 data points of the origina1 data and山ecurve generated by出cdesign variables. Thc design variables arc出everticωiIIustratcd in the figurc. The basic curve of Figu陀 9.t3 is obtained using由巳

2.5

2

1.5 a .. 帽司

>0

0.5・・・ 6

。

-0.5 0

:。

0.5

Flgure9.13

。

。

1.5 2 2.5

x data

Best fltatg自neratlont"， 1.

374 GLOBAL OPTIMIZAnON 9.2 NUMERICAL TECHNIQUES AND ADDITIONAL EXAMPLES 375

司g咽 0.5

=均0¥-" "...，. 司咽咽 ..咽 4ドー"""'，・ .，....，....，圃 . 唱・‘....'咽‘・+ 。、.

-0.5

-1

。嗣 ZO L

自立 0.4 0.6 O.目官 1.2 1.4 1.6 1.8 2 x data

Flgure 9.14 Best fit: generation t= 200.

35

30

2s

a

u

P

3

・4

・・

』白』』由唱@』咽ヨぽ帥

10

に一。唱。。唱50 200 250

x data

円gure9.15 Objeclive function for every generation.

>> DrawCurve(Xbest(1，1:8))

血ispr'田吋ure.1t is possible to scparately accumulate only designs白紙 arefeasible

while continuing to implement the standard procedure. Sccond， use a strategy to命ive

the search process through constraint satisfaction and decreasing出eobjcctivc， simultaneously or separately. Thc algorithms indicatcd in this chapter can easily incorporatc出巴n田 dedch叩 ges.

The a1gorithmic procedure as implemented do田 notreal1y care if也eproblem

is continuous or discrete. 1t just works on a current set of values for the design variables. Discrete va1ues wil1 primarily require minor changes in thc way the population is generated. A1so thc implcmcntation of the crossover and mutation operators wil1 need to be changed. These changes in no way detract from the m吋or

idea bchind the GA.

D悶 wCurve.mfunction. To draw the initia1 curve (出efigure is edited subsequently

with plotedit) in the Command window:

Figure 9.14 is the best design at出efina1 generation (generation number = 2∞). 百leobjcctive function is 0.0729.1t is clear from the figure白紙出巴 GAis attempting

to find the solution. This can be established by I∞king at Figure 9.15 which is山e

iterationlgeneration history of the GA for this example. The objcctive function is

genera11y dccreasing through出egenerations. Figur芭9.16is a sca1ed representation to display the behavior of the GA at lat巴riterations. From白edata it is al80 apparent thnt

出iscxamplc needs to run for morc白叩200generations. The fina1 figure， Figure 9.17 represents白巴 movcmentand the clustering of the vertices through the generations. It

is suggestivc of loca1 refining白紙 isassociated with the GA.

Suggested Extensions: Most of the engin館 ringdesign problems involvc

cons町高ints.Currcntly the most popular way to handle constraints is to reformulate thc

problem using pcna1ty functions (Chapter 7).τ'here are scvcral oth巴raltematives to

Closlng Comments: In summary， for a heuristic techniquc，出eGA is very

imp陀ssive.百lCsamc can be said for出巴 SA.Including the fact白atpractical

engineering problems must require partly disαete rep民sentation，血etcchniques in

Chapte隠 8佃 d9 are very relevant to出epracti田 ofdesign and development. With

the enormous computationa1 resou民ein白edesktops and laptops， an effectivc programming language Iike MATLAB， and a wide varicty of algorithms山atarc available， it is possiblc now to design an individual approach to many optimization


0.5

0.45

0.4 scaled for lowest values

0.35

ロ

お

u

t

-

t

nu

』

Oヒ也事

e咽

2Z

乞同由由〉

』。由コ個〉

。〉崎

-0.5

-1

-1.5

-20」.4 0.6 0.8

0.15

0.1

0.05

。。

REFERENCES 377

Locatlon of deslgn varlables over 200 generations

60 80 100 120 140 160 180 200 x data

2.2 2.4 20 40 1.2 1.4 1.6 1.8 x value of vertex

2

Flgure 9.16 Ob同tivefunctlon scaled 10 show contlnual decrease lor laler generatlon. 円gure9.17 Trace 01 Ihe deslgn varlables Ihrough generatfon.

REFERENCES problems. This is important to practicing engineers who can bring inluition and insight for the particular problem白eyhave to solve in a tight time frame. In the engineering curriculur丸紅aditionaldesign optimizalion courses lend to deal only wi出deteロ凶凶sticcontinuous algorithms. That in itself was mo民自叩 thetime available for instruction. Disc問 teoptimization or global optimization are neeessarily skills l回 medby personal endeavor.百lCovercmphasis on deterministic programming needs to be revisited in the light of current develop即時in白earca of optimization. T叫ay，computer programming in engin田 ringdepar加 entsand its reinforccment is usually adjuncl 10 other developments in血ecurriculum. New programming platforms Iike MATLAB offer a way to resolvc出isdisadvanlage. New algorithms also reflecl a simplicity that can be回 sily官加slatedand applied.

百 isbook has been about programming in MATLAB， leaming about traditional and current techniques in design optimization， and transloting these optimization

蹴加iquesinto appli回 .tions.lt is also about exploration and developing an insight凶o

the process of design optimization. The code白紙 isprovided is only meant to be a

starting point.

1. Arora， J. S.，lntroduction 10 Optimal Design， McGraw-HiII， New York， 1989.

2.K討'kpa凶ck，S.， Ge)att， G. C.，lInd Vecchi， M. P.， Optimization by Simulated Annea1ing， Science， Vol. 220， No. 4598， May 1983.

3. Me凶 polis，N.， Rosenbluth， A.， Rosenbluth， M.， Telle， A.， and Teller， E.， Equlltions of Sate Ca1culations by Fast Computing Machines， Joumal of Chemical Physics， Vol. 21， pp. 1087-1092， 1953.

4. Bochavesky， 1. 0.， Johnson， M. E" and Myron， L. S.， Generalized S回 ula除dAnnea1ing for Function Optimization， Teclmometrics， Vol. 28， No. 3， 1986.

5. Jones， A. E. W.， and Forbes， G. W.， An Adaptive Simulated Annea1ing Algorithm for Global Optimization over Continuous Variables， Jour:聞 1ofGlobal Oplimil.Cltion， Vol. 6， pp. 1-37，1995.

6. Cohn， H.， and Fielding， M.， Simulated Annealing: Searching for an Optima1 Tempe凶 ureSchedule， SIAM Joumal on Optimization， VoL 9， No. 3， pp. 779-782， 1999.

7. Holland， J. H.， Outline of a Logical Theory of Adaptive Systems， Joumal of ACM， Vol. 3，pp.297-314，1962.


8. Dumitrescu， D.， Lazzerini， B.， Jain， L. C.， and Dumitrcscu， A.， Evolutionary Computation， Intemationa1 Series on Computationa1 Intelligence， L. C. Jain， (edふCRC

Press， Boca Raton， FL， 2000.

9. Gen， M.， and Cheng， R.， Genetic Algorithms and Engineering Optimization， Wiley Series in Engineering Design and Automation， H. R.， Parsaci (scrics cd.)， Wilcy-Intcrscicnce， New York， 2000.

10. Goldbc屯， D.， Genetic Algorithms UI Search， Optimizarion， and Machine LeamUlg， Addison-Wesley， R伺 dingMA， 1989.

11. Houck， C. R.， Joines， J. A.， and Kay， M. G.， A Genetic Algorithm for Function Optimization， p句eraccompanying the GAOT (Genetic Algorithm Optimization T∞Ibox)， No巾 CarolinaState Univeぉity.

PROBLEMS

No fonnal problems are suggested.百lereader is encouraged to develop his own

variants of批 twoalgorithms， including fonnal ones available in出esugges削

literature. He is also encouraged to consider extensions to constrained and discrete

problems. He is recommended to use his versions to explo問 variousexamples found

in the literature. In comparison with出edetenninistic techniques (until Chapter 8)， thes巴 heuristicmethods are truly fascinating and il1ustrate considerable robustness， and what is cven more challenging 10 our pcrception， they can solve any problem with no reservations whatsoever.τhey are incredibly basic in construction， simple to program， and easy to deploy. They only require a lot of time to search the design spa田.

司"F J

10

OPTIMIZATION TOOLBOX FROM MATLAB

百leMA11.ABfi創凶lyof producls includes the Optimization Toolbox， which is a librarv

ofm・filω 蜘伺.nbe used to叫 vep帥 lemsin脚蹴aof optimiz副on.1tcon凶-many of the methods that have been explored in出isbook， and several白atare not

available in the Toolbox. For山os巴consideringthe practice of design optimization

seriously， the Optimization Toolbox is a worthy investment. Many ofthe techniques

and discussions in the book were kept simple to il1ustrate白巴con田，pts，回d出em-files

were used to provide MA11.AB programming experience. Most of the m-files available

on曲ew中山 willneed to be d附 lopedfurther for scrious prac削 application.Another important reason for having access to白eOptimization Toolbox is白紙 it

allows one 10 take advantage of MA 11.AB' S open source policy，也atis， to have低 cess

10 the source files. You may be able to modify and extend those filesωmeet your special optimization 問 quir芭ments.While some of the programs from the Toolbox

have b田 nused or referenced elsewhere in曲isbook，出ischapter collects them in a

single place for回 syreference. As can be expected， the material of出ischapter will

be dated.τ'he chapter is based on MA11.AB Version 5.3. As of白iswriting， Version 6.0f6.1was shipping.In general，the MATtAB products，especially the Optimization Toolbox， have reached a level of maturity such 伽 t newer versions s訂臼h削o

invalidate the infonnation in 山刷i胞ss釦ec叫“ωo叩n.What should be exp 田 ted， however， is 伽 t

the material in白ischapter wi11 most likely be upgraded， enhanced， and extended with each new rele凶 eofMA札油.

角7Q

380 OPTIMIZATION TOOLBOX FROM MATI.AIl

10.1 THE OPTlMIZATlON TOOLBOX

Most of the basic information in出ischapter is from山 user'sguide for the Optimization Too1box [1]. Ifyou are buying a specia1ly bundled CO that includes the Optimization Toolbox， then you may not have Reference 1 in book fonn. You may have Reference 1 as a POF file on thc CO or you may have access to it. For exnmple， on山eWindows NT platform for dcfault dircctory insta11ation the POF version of Reference 1 is avai1ablc at

C:¥MA TLABR 11 ¥help¥pdCdoc¥optim¥optim_tb.pdf

Refcrence 1 providcs examplcs of usc， discusses sp田 ia1cases， a1gori山ms凶 ed，阻dmany 0出巴rsignificant topics.τ五ischapter does not attempt to document them. Limi飽dduplica品onis凶 edto bring attention to the most useful fl回，tures.τbeemphωis is on i1lustrating the use of出eToolbox for various examples in位。du田d泊

the book. In fact， a lnrge number of the techniques in th巴Toolboxare not used in血ebook and are not covered in白issection. The experience provided in this chapter should be sufficient to understand and use the new ones cぽectivcly.

To use出cprograms or m.filcs in the Toolbox， thre巴importantactions must bc initiated. One， formulnte山eproblem in the format expccted by MATLAB with悶 pcctto山cparticular function thnt will bc calIed. Most1y，出iswill involv巴 usingapprop由teprocedures to sct the right.hand side of the constrnints to zero or constant va1ues. The Toolbox a1so makes a point of distinguishing betwecn linear and nonlinear constraints. S巴cond，set/change白eparamete凶 for白eop白nizntiona1gorithms. Thi吋， use the appropriate optimizer and ensure白紙 ava1id solution has bcen obtained. AII of白巴seactions natura1ly depend on the actua1 mathcmatical model of出eproblcm.

市0.1.1 Programs

Thc different types of optimization problems出atcan bc solved wi出 thcversion of the Toolbox installed on白巴 computercan be discovered in the Command window， Help window， or in the browser using Help Oesk. In出eMATLAB Command window type

>>help optim

The output is reproduc巴dbelow. They are a11 of the various m.files in the Toolbox and their optimization function is a1so idcntified.

Optimization Toolbox. Version 2.0 (Rl1) 09・Oct・1998

What's new. Readme -New features， bug fixes， and changes in this

version.

10.1 THE OPTIMIZATION TOOlBOX 381

Non1inear minimization of functions.

faiabad--Scalar bounded nonlinear Eunction minimization.

mi:22::…Mulωimen山叫 constrained nonlinear

定minsearch-Multidimensionalunconstrainednonlinear minimization，by Nelder-Mead direct search method .

fmiaunc-Multidimensional unconstrained nonlinear minimization.

fgaminf-MIll tidimensional cons trained minimi zation， semi-infinite constraints.

Nonlinear minimization of multi-objecヒivefunctions. fgoa1at?ain-Multidime悶 iona1 goa1 attainment

optimization.

まminimax・ Mu1tidimensiona1minimax optimization.

Linear 1east squares (of matrix prob1ems) .

1aqlin-Linea?leastsquares with linear constraints. ∞::ご:t::-LLn町 least 叩 ares with non明 ativity

Nonlinear least squares (of functions).

?sqcuwefit-Nonlinear curvefitting via least squares (wエthbo山 lds).

lsqnonlin -Nonlinear least squares with upper and lower bounds.

Non1inear zero finding (equation solving). ~zero -Scalar nonlinear zero finding.

fsolve -Non1inear system of equations s01ve function Solve.

Minimization of matrix problems. 1inprog -Linear programming・

quadprog -Quadratic programming.

Controlling defaults and options.

optimset Create or a1ter optimization OPT工ONSstructure.

optimget - Get optimization parameters from OPTIONS structure.

Demonstrations of large-sca1e methods.

circusteat-Quadratic programming tofind Shape of a circusヒent.

molecule-MOlecule conformat ion SOILltion using . unconstrained nonlinear minimization.

382 OPTlMIZATION TOOLBOX FROM MAn姐

optdeblur 工mage deblurring using bounded linear

least-squares.

Demonstrations of medium-scale methods.

optdemo -Demonstration menu. tutdemo -Tutorial walk-through. bandemo -Minimization of banana function.

goaldemo -Goal attainment. dfildemo - Finite-precision filter design (requires

Signal Processing Toolbox) . datdemo -Fitting data to a curve.

The tutdemo is recommendcd for a demonstration of印刷uresof山 Tool胤Of the above items， optimset， linprog， quadprog， fminunc， and fmincon are further explored in由ischapter.

A clearer picture of the capability of血efunctions can be obtained from the tabular

graphi叫 rcpres側 a伽 1of thc P帥 lemscts in the first chapler of Reference 1. For

example，出efour tcchniques referenced in由ischapter are iIlustrated in Table 10.1.

10.1.2 Uslng Programs

Eacb of thc functions in the Toolbox白紙 implementsan optimizatios tcchllique can be凶 edin sev釘 always.τ'hese are documented in Reference 1 and are also available

online through one of the mechanisms of ob也ininghelp in MATLAB・Forexample， typMghelpmneonin thCommandwindowidentifies thefollowingdiffemtways

ofusing由巳 functionlprogram(the following is edited to show only types of usage).

FMXNCON Finds the constrained minimum of a function of

several variables. FM工NCONsolves problems of the form:

min F(X) subject to: A*X <= B， Aeq*X Beq (linear constraints)

Table 10.1 Graphical D笛 criptionor Some Toolbox Functions

Typ巴

Uncons町aincdminimizauon

Linear programming

Quadrauc programming

Cons釘泊nedminimizauon

Notation

min.f(x)

min.fT x such that

[A][μx]壬[b];[A叫][μx]=[b叫]; l~x 当 u

m吋凶T刊l剛+ハ叫白

[μAJ[x] 三[股b];[μA.J[μ;x] = [.股b.qlよ壬壬xSum官ni加n.f(x:功);such 凶a必t c(x功)~O伐 Ceq(X:吟)=0

[AJ[.μ[X]心]S [b); [A.叫qUμ凶x]=[b叫]; l~亘 xSu

funcuon

fminunc， fminsearch

linprog

quadprog

fmincon

10.1 THE OPTIMIZATION TOOLBOX 383

C(X) <= 0， Ceq(X) 0 (nonlinear constraints)

LB <= X <= UB

X=FMINCON(FUN，XO，A，B) X=FMINCON(FUN，XO，A，B，Aeq，Beq) X=FMINCON(FUN，XO，A，B，Aeq，Beq，LB，UB X=FMINCON(FUN，XO，A，B，Aeq，Beq，LB，UB，NO恥 CON)

X=FMINCON(FUN，XO，A，B，Aeq，Beq，LB，UB，NONLCON，OPT工ONS)X=FMINCON(FUN，XO，A，B，Aeq，Beq，LB，UB，NONLCON，OPTIONS， • Pl，P2，... ) [X，FVAL]=FMINCON(FUN，XO，.. .) [X，FVAL，EXITFLAG]=FMINCON(FUN，XO， ...) [X，FVAL，EXITFLAG，OUTPUT]=FMINCON(FUN，XO，...) [X，FVAL，EX工TFLAG，OUTPUT，LAMBDA]=FMINCON(FUN，XO，...)(X，FVAL，EX工TFLAG，OUTPUT，LAMBDA，GRAD]=FMINCON (FUN，XO ， ...) [X， FVAL， EX工TFLAG，OUTPUT ， LA旭 DA，GRAD，HESSIAN]=FMINCONFUN，XO，...) .

There蹴 12ways of山崎 theρninconfunction.百1Cdi仔erenωsdepend on血e

information in the mathcmatical model白atis bcing usoo and the postoptimization information that should be displayed. In using tbe differcnt釦nctioncalls，叩ymissing data 白atdo not exist for the particu1ar examples must be indiωtOO using a null vector (0).

τbere are由民eke戸vord旨加 theabove list of function calls白atneed special

altention in cons回 inOOnonlinear optimization.

EXITFLAG: For fmincon血isindica飽sif thc optimization was successful.

If EXITFLAG is: >0 then FMINCON converged to a solution X.

o then the maximum number of function evaluations was

reached. < 0 then FMINCON did not converge to a solution.

OUTPUT:羽山 isspecia1 MATLAB structure element. Structure is a programming cons凶 ctusOO in many higher-Ievellanguages. It is simi1ar to array in MATLAB. An array will contain only simi加 elementsー ukcall numbcrs or all strings. A s凶 cture，on the other hand， can contain disparate elements， such as severa1 numbers and strings. The elements of由estructure can be accessed by using the name of the 5回 C組問

element followed by a period and出CPI吋 efinOOkeyword for the particular elcment. For cxample， two of the elements of the OUTPUT are:

OUTPUT. i terations: the number of function cvaluations OUTPUT.algorithm: thenameof出ealgorithm usOO to solve the problcm

LAMBDA I This is al50 another structure in MATLAB and contains肱 Lagrange

m叫tipuers.百1evaluesof由民esmultipliers are useful for veri今ingthe satisfaction of

血enecessary conditions of optimization. The values can be recover叫 by山esame

pr田吋山'Cindicated for thc OUTPUT structurc.

-圃

384 OPTIMIZAnON TOOLBOX FROM MAn凋

LAMBDA .lower: value of the multipliers at the lower bound LAMBDA. ineqnonlin : value of the multipliers tied to the inequality

cons町剖n悩(s'sin山巴加ok)

Thesc items werc associated with the functionρnincon. Other functions may havc differentladditional type喧ofinformation availabl巴.

10.1.3 Setting Optlmization Parameters

There ar芭 severalparametcrs that are used to control the application of a particular optimi四 tiontechnique. Thcse parameters are available through thc OPTIONS struclure. AIl ofthem are provided with default values. Thcre will be occasions when it is n巴cessaryto change the default values. One example may be出atyour constraint tolerances are required to be lower th姐 thedefault Or EXITFLAG recommends a larger number of function evaluations to solve白E戸oblem.These p紅白metersare set

using a name/value p瓜r.Gencrally白ename is usually a string whilc the value is typically a numb巴r.The list of parameters is available in Reference 1 and can be accessed most easily in the Command window by typing

>> help optimset

A partiallist of the parameters in the OPTIONS structure， copied from the Command

window. follows.

OPTIMSET PARAMETERS Diagnostics ・ print diagnostic information about the

function to be minimized or solved [ on I {off}l DiffMaxChange -Maximum change in variables for finite

difference gradients [ positive scalar I .{le-l} 1 DiffMinChange -Minimum change in variables for finite

difference gradients [ positive scalar I {le・S} 1 Display -Level of display [ off I iter I {final} ] GradConstr - Gradients for the nonlinear constraints

defined by user [ on I {off} 1 GradObj - Gradient(s) for the objective function(s)

defined by user [ on I (off}l Hessian -Hessian for the objective function defined by

user [ on I {off} ] HessUpdate -Quasi-Newton updating scheme [ {bfgs} I dfp

I gillmurray I steepdesc 1 LineSearchType Line search algorithm choice

cubicpoly I {quadcubic} 1 MaxFunEvals - Maximum number of function evaluations

allowed [ positive integer 1 Maxlter Maximum number of iterations allowed

positive integer ]

10.2 EXAMPLES 385

TolCon Termination tolerance on the constraint violation [ positive scalar J

TolFun.・ Terminationtolerance on the function value positive scalar J

TolX -Termination tolerance on X [ positive scalar 1

The parameter names are identified in bold fon1. These parameters are the ones白紙

wi11 con回，1a standard optirnization application. The default values for肱 parame防rare given within the curly braces. For example:

>> opヒions opヒimset(‘Diagnostics'，'on' ，'Display'， 'iter' ，'Maxlter' ，'500' ，'HessUpdate' ，'dfp' ，'TolCon'，'l. Oe-OS')

U the above statemcnt is used prior to actually calling an optimization technique，血edefault values for those p釘倒neterswi11 be changed in the OPTIONS structure. The default vaJues for a particular technique can be found by typing the Command window

>> options opヒimset(‘fmincon')

前lere創芭 severalothcr ways to use the optimset program itself. For details consult Reference 1.

10.2 EXAMPLES

百1巴 examplesi11us町atedin出issection are continuous optimization problems discussed in山ebook.百leyare from Iinear programming. quadratic programming. unconstrained minimization. and constrained minimizatioo. Only the appropriate mathematical model is iotroduced prior to i11us回 .tingthe凶 eof the Toolbox function.

10.2.1 Linear Programmlng

The LP example uSed here includes all three types of constraints. The toolbox function is linprog.

Example 10.1

Maximizc z = 4xI + 6x2 + 7X3 + 8x4

Subjectto: XI + X2 + x) + X4 = 950

X4~ 400

2x1 + 3X2 + 4X3 + 7X4 S 4600

3xI +4x:け 5X3+ 6x4 S 5000

Xlo X2. X3. X4 ;?; 0

386 OPT1M1ZAT10N TOOLBOX FAOM MATI.AIl

百lelinprog program solves the following problem:

Minimize fTx; subject to [A](x] S [b]

The objective function for the examplc needs to b巴 changedto a minimization

problem. The greater白佃orequal to constraint needs to be changed to a less山卸 orcqual to onc. The linprog program a1lows equality constraint佃 dexplicit bounds on

variables.百leequality cons釘aintcan be set up with A句andb句・Thereare not many p紅白臨むers白atneed to be changed except switching of the 'LargeScalc' problem to

・off.'For only four design variablcs Icaving it on should be permissible. The form of linprog that will be used is

[X，FVAL，EXITFLAG] LINPROG(f，A，b，Aeq，beq，LB，UB)

This form is not explicit1y documented in Reference 1 or in the help documentation.

Ex10_1.m1 is出em-ftle that is uscd to set up and run the problem. It is not displayed

here but running the file in自己Commandwindow produces th巳followingoutput

Optimization terminated successfully. X

0.0000 400.0000 150.0000 400.0000

FVAL

-6.6500e+003 EXITFLAG =<

1

The solution to出巴 problemis

x; = 0; X; =400; x; = 150; x; =4∞7."=6650

10.2.2 Quadratic Programmlng

Quadratic programming refers to a problcm with a quadratic objective function and Iinear constraints. While such a probl巴mcan be solved using nonlinear optimization t巴chniquesof Chapter 7. special qua命冨ticprogramming a1gorithms based on the tech凶quesof Iinear programming are normally出 edas出eyare simple and faster. If the objectivc function is a quadratic function in the variables， then it can be cxpressed inqua的自form.For examplc，

f(x1・ろ)=x1+2x2 +寸+列X2+~

can be cxpressed悩

I Filcs 10 be downlonded from thc web sile nrc indicalcd by boldface岨nsserif type.

10.2 EXAMPLES 387

川)= [1 2]同+ド1 x2] [i !][::J

M

f(.日 )= [c]T [x]中.X]T間 [x]

Example 10.2 Solve the qua合aticprogramming problem 「 3

M刷in凶n凶im山n凶由nize也邸i包回ze 仇い)川=斗[一1ト円一-2一3]T州T

Subject ω x刻1+2x勾2一x勾3壬2.5 L 1

2x1 -X2 + 3X3 = 1

X1. X2' x) ~ 0

-1 11

2 ll[x]

1 41

The program to be used from the Toolbox is quadprog. The standard format for this

program 18

Minimize f(x) = cT X + ~ xT胎

Su同ectto: AxSb

A句 x=b.句

LBSxS UB

The solution can be found in Ex10_2.m. The default optimization p直前田:tersare

used so山atthe options structure does not havc to be changed. The acωaI calling

statement used is

[x， fval，EX工TFLAG] quadprog(H，c，A，b，Aeq，beq，LB，UB)

The solution is formatted and displayed in thc Command window.1t is copied from

the Command window:

Optimization terminated successfully.

Final Values optimum Design Variables

一一一一一ーーーー・ーーーーー------ーーーーーー一一一一一一

0.5818 1.1182 0.3182

Optimum function value

-1. 9218

担

388 OPTIMIZATION TOOLBOX FAOM MATWI

Lagrange Multipliers for inequality constraint

0.0218 工nequalityconstraint

2.5000 Lagrange Multipliers for equality constraint

0.0164 Equality constraint

1.0000

10.2.3 Unconstralned Optimization

Example 10.3 is used 10 ilIustrale山巴 uscof fminunc，山eprogram to be山 edfor unconstrained minimization. 11 is a simple curv巴自民ingexamplc (a 101 of

unconstrained minimization problems in engineering are curve fitting used for developing correlation from empirical data). The Toolbox has programs specifically for curve fitting. It may be a useful idea for the reader to compare the solutions using thos巴programs凶白isprovides an opportunity to use programs from由eToolbox not illus回 tedin this chapter.

Extunple 10.3 A set of data over 100 points represen出 ga cosine function isα四 ted.A qua世aticpolynomial is used 10 fit the data. An unconstrained minimization problem based on the sum of squ紅白 ofthe error belween original data and the polynomial， over all出edata points is solved for the coefficients of the polynomial.

百由isan ex創npleof the classicalleast sq凶 rede町。.rproblem.The example is solved in曲巴m-fiIeEx1 0_3.m. Running thc example also displays

a comparison of the fit on a figure which is not included here. Two solutions are obtained in the example starting from the same initial guess. The first one is solved using凶 'er-s叩tpliedgradie臨.The second is the default finite difference 伊 dients.The results ar芭 copiedfrom the Command window:

Final Values ・ Oserspecified Gradients Optimum Design Variables

-0.4309 -0.0365 1.0033 Optimum function value

2.4055e-004 Gradients of the function

1.0e-007* ー0.1324 -0.2198 -0.3275

わ一一

10.2 EXAMPLES 389

Fiaalvalue--Default Finite Difference Gradients Optimum Design Variables ーー，ーーーー・・ーーーーー一一一ーーー--ーーー一一一一一ーーーー

ー0.4309 -0.0365 1.0033 Optimum function value ーーー・ーー・司・・ーーーー一一一一ーーーー・ー.ー

2.4055e-004

Gradients of the function ーーーーーーーーー一一一一一一ーーーーー・ーーー・ーーー一一←一一ーーーーーーー

1.0e-004 *

-0.0121 -0.0270 0.5715

It is generally belie叫出鉱山制叫叫附 ofgradients wouJd Jead 10 a fiぉterand better叫 ution.For ti中nuncto附恥gradientscomputed by山町，changcs havc to bc made to山eoptions s凶 clu隠悩 wellas the file wh巴問 theobjective function is c蜘 i削(油IJ10_3・m).In印刷onsstruct脱出e'OradObj' parameter should bc sct to 'on・'inISISEmponmt Otherwise MATUB dom not cxpECt thE gradients to bE・

provid吋 bythe user. The option for this ex創npleis sel by

TZ;;??s、TolFun'，マ;?と;J:??三;;zp;;;;ff;，、山play'， τbis causes出epro伊 mto look for恥伊dien岱泊出esecond elem則 oftheretum value.The functIon Statement usdfor obj10」3.mis

function [f， grad] objlO_3(x)

Bo白山eo対cctivefunctionの叩d出egradie附 (grad)are calculated in the same function m-fiIe.

10.2.4 Constrained Optimizatlon

Example 10.4 The example is the same as Example 8.4 and陀P児sentslhe constrained ex創nplefor this scction.τne mathematical model is

Minimize f(X) = pLAc

Subject to: g.(X): WLJ - 12EI S 0.25 (dcfl田 tion)

g2(X): WLx. -8/1σy S 0 (normal stress)

g3(X): WQc -21xJ'ty S 0 (shear slress)

Some gcomctric constraints to relutc the various design variubles are

390 OPTIM(ZATION TOOLBOX FROM MAn.AB

g4(X): x， -3X2壬0

gs(X): 2x2 -x( S 0

g6(X): X3 -1.5x4 S 0

g7(X): 0.5x4 -x3 S 0

The side constraints on the design variables are

3 Sx( S20， 2Sx2S 15， 0.125 SX3 SO.75， 0.25 SX4 S 1.25

The foIlowing relations define the cross-sectional area (Ac)' moment of inertia (乃，andfirst moment of area about the centroid (Qc):

Ac = 2x2x4 + x(xヨーZちら

Iーらxt(ろ-xJ(x，-2xi一一一12 12

Qc=0.5.引いl一川+0.5x向 _XJ2

The parame旬rsfor出巴 problemar官 w=1.4場2000，L = 96， E = 29 E+06， P = 0.284， σ'y=36E+03，τ:y = 21 E+03. There are three nonlinear inequality constraints and four linear inequality constraints.官lecode for running the prograrn is available in Ex10_4.m.

The program to be used is frnincon. The following OPTION par副netersare changed.

options optimset(‘LargeScale' ，‘off' ，‘Display' ， ‘iter' ，‘TolFun' ， 1. Oe・08);

百lecalling statement used for fmincon with the objective function available in objective.m， and the nonlinear constraints in nonlin_cons.m， is

[x， fval， exitflag， output， lambda] fmincon(‘objective' ，xO，A，B，日，口，LB，UB，‘nonlin_cons'，

options);

The Iinear inequality constraints are set up in the matrix A and vector B. There are no Iinear equality constraints and they are indicated by the nuIl matriccs. The lower and upper bounds are applied through the vectors LB and UB. The customized part of山eoutput displayed in the Command window is included below. Accessing出巴 variousStructure values will provide details about the performance of白巴 algori出m.

Final Values

Optimum Design Variables -ーーーー一一ー一ーー一ー一一一一ーーー・ー - 司 -- --ーーー・・ー

6.8671 2.2890 0.1802 0.2500 Optimum function value ー.ーーーーーーー一一一一一一一一一ーーーーーー司・・ー噂，

62.4905

Final Nonlinear Constraints - --ーーーーーー一一一一一一一一ーーーーーー--ー

1.0e+009 合

-2.7304 -0.0025 -0.0001

REFERENCE 391

Lagrange Multipliers for Nonlinear constrainヒsーー・・ーーーーーー一一一一ーーー・・ーーーーーーーー.ーーーーーーーー一一一一一一ーーーーーーーー.

000

Lagrange Multipliers for Linear constraints ーーーーーーーーーー一一一ーーー・ーーーー---

2.5879 0 0 0

The reader is encouraged to run the example with ・Diagnostics'on， explore恥opttons structure， and山eoutput s釘ucture.

REFERENCE

1.白lmmJ-R.BmEI仇 M.A.， and Grnce， A.， Optimization Toolbox-for Use witlr MATLAB穴User'sGuide， Version 2， MathWorks Inc・， 1999.

理liiIi画園

Abstract mathematica1 model， 5， 9-10 Acceptable solutions， 17 Active constraint set， 293 Acrospace design， 1 Algorithms，21 ALM mcthod， see Augmcnlcd Lagmnge

Multiplicr method Ana1ytica1 conditions， 175-176 Annealing 8chcdulc， 359 Anificia1 var泊bles，98 Augmented Lagmnge Multiplicr (ALM)

m官thod，276-281 algori山m，277-278application， 278-281 unconstr凶nedproblem，

位unsform瓜ion10， 276-277 Augmented matrix， 103-104

Bez.ier parame凶ccurvcs， fitting of， 258-262

BFOS method， see Broydon-Fletcher-Ooldfarb-Shanno me曲。d

Biscction tcchnique， 209-210 Bmnch and Bound

叫gori出m，333applica山 n，333-335tcchniquc， 329田 336

Broydon-F1etchcr-Goldfarb-Shanno (BFOS) method

n1gorithm. 250 application. 250-25 1

Cartωian spaces. 106

INDEX

C (Ianguage)， 24 Classωofproblcms.21 Computationa1 rcsources. 2 印刷gateOmdicnt mcthod. 244-246，

255 algorithm， 244-245 application， 245-246

Constmined optimization problems. 265-315，389-391

Augmentcd Lagmnge Multiplier (ALM) mcthod for solving，

276-281 beam design， 310-312 control， optima1， 313-315 dcfinition of problcm. 266 曲 ectmethods for solving， 281-302

• Extcrior Pcnnlty Function但PF)method for solving， 271-276

flagpole problem， 307-310 formulation ofproblem， 266-267 Gencrnlized Reduced Gmdient (GRG)

me伽 dfor solving， 297-3但generic a1gorithm for， 269-270 indircct methods for solving， 270-281 目白ω鈍 ryconditions for， 267-269 S叫uentia1Gradient R凶 tomtion

a1gorithm (SGRA)， 3但ー307S叫ucntiaJLinear Progmmming

(SLP)叫gori出m，284-289S叫uentialQuadmtic Programming

(SQP) a1gori山m，289-296Cons位置intfunctions， 6， 7 Cons位泊nts，1，5

393

394 INOEX

Cons位凶n匂 (conrinued) 伺uality.7-8inequality. 8 in LP prob1ems. 98 in NLPprob1ems. 157-159.172-175.

179-181 side.8

Continuous re1axalion. 321-322 Conlinuous vnriablcs. 20 Con回 1.0p山nal， 313-315 Convcrging 10 thc solution. 21 Convcx functions. 352-353 Corners.20

Davidon-Flclcher-PoweU (DFP) method. 246-249

Dcgrees of freedom (00町.104Dcscent funclion. 292 D凶 ign.1.2Dcsign functions. 5-6 Dcsigno句eClivcs.5Dcsign pammelcrs. 5 Design space. 9 Design variab1cs. 4-5 Deslgn veclor. 5 DfP method. see Davidon-Fletcher-Powell

me曲。dDigraphs. 336 Discrete optimiz瓜ionprob1ems (DP).

318-348 Bmnch nnd Bound lechnique for solving.

329-336 continuous re1axationωapproach for

solving.321-322 discrete dynamic programming ns

I民 hniquefor solving. 336-341 Exhnustive Enumeration techniqu巴for

solving. 326-329 I・民監ndcsign.341-343slnnd釘dapproach to solving. 322-324 Zero-Onc Integcr Programming (ZIP).

343四 348Di鉱偲teprogramming. 20 Discrete varinblcs. 20 DOF (degrees of frecdom). 104 DP. see Discretc optim包ationprob1cms;

Dynamic progrnmming Dual problcm. 138-148

Dynamic programming (DP). 336-341 algori白m，340functional句U凶onfor. 338-339

EBV (Enlering Basic Variablc). 118 Editor (MATLAB). 33-37 Enginee巾Ig.2Entering Basic Variab1e (EBV)， 118 EPF mcthod， see Extcrior Penalty Function

method Equality ∞nstraints.7-8 Euclidcnn spaccs. ，トdime目sionll1.1O晶

Exhaustive EnumcraLion techniquc. 326-329 algorithm. 327 applicalion.327-329

Exterior Penalty Function (EPF) method， 271-276

algorithm.272-274 appliclllIon.274-276

Fcasib1e domain. 23 Feasib1e solutions. 17 First-order co目ditions(FOC). 176-178 First-orderllincar variation. 169 Flagpo1e prob1em. 307-310 FOC. see First-order conditions Formu1ation of prob1em， see Prob1em

fonn叫8!tionFORTRAN.24.227 Function(s):

constraint， 6. 7 objectivc，6 ofone variuble. 162-164. 169-170 oftwov副 ab1cs.56-63.164-171

GA. see Genctic A1gorithm Guuss-Jordan c1imination. 107 Generalized Reduecd Gmdicnt (GRG)

me曲。d，297-302 a1gorithm. 299-300 upplicalion. 3αト302stepsize. ca1cu1alion of. 299

Generic Genetic A1gori出m(GGA).367-370

Gcnctic A1gorilhm (GA). 350. 366-377 gcneric fonn. 367-370 suggcsled extensions 10. 374. 375

G10bal optimization. 350-377

definition ofproblcm. 351-357 Genetic A1gorilhm (GA) for出 ein.

366-377 minimum， global. 351-353 numcrical techniqucs for. 356-358 Simuluted Annealing (SA). 358-366 solution. global. 354-356

Golden section method. 214-217. 219-220. 223-225

Gmphica1 optimiza由民 16-18.45-92definition of problem. 45-48 nnd format for gmphical display. 47-48 function of two variables. 56-63 nnd graphica1 user interface. 81-91 wi白HnndlcGniphics， 79-81 恥attransfer examp1e. 73-79 solution.伊 lphi伺 1.48-56structural engineering design examp1e.

64-73 Graphical solutions:

linear progrnmming (LP) prob1ems. ¥07-1¥5

nonlinear progrnmming (NLP) prob1ems. 171-175

one-dimensional problcms. 205-206 Gmphical User Interface (GUI). 24. 48. 49，

79.81-91 GRG method， see Gencralized Reduced

Gmdient method GUI. see Gmphicll1 User Interfacc

Handle Gmphics. 79-81 HnndJcs.49 Heat釘ansfer.73-79

In伺ualitycons回 ints.8Infinite solulions， 17 lnitial d民ignsolutions. 21 Jnleger progr佃 lming.20IlIteger variab1cs. 20 llcrative tecbniqucs. 21

Kuhn-Tucker conditions. 192-193. 196-200，267-269

Lagrnnge mu1tipliers. 180-192 LAMBDA progmm (Op白川Z8tionTI∞1box).

383-384

INOEX 395

Leaving Basic Variab1e (LBV). 118 Lineard叩宮ndence，¥01 Linear progmmming (LP) prob1cms. 23-24.

93-153 cons住凶ntsin， 98 delerminants泊.¥02-104 dual prob1em associa包dwi白.¥38-148 cquality consb冨ints，exnmple involvlng.

130-134 cx師 .plcsof. 11ι111， 124-138.

385-386 four-variable prob1cms， 134-138 graphical Solulions of. ¥07-115 infmite solution of. 114 MATLAB， solution using， 120-124 modcling issucs in，98-107 negative d邸ignvariablcs in. 98 NLP problems vs.. 20， 93-94 nosol凶onfor. 114-115 prim叫problem.138 sensitivity nnalysis wi山.148-151Simplcx method forsolving， 115-124 stllndard format for. 94-98 transponalion prob1em ex剖np1e.

124-130 unbounded solution of， 114 unique solution of. 114 山首tvωtorsi民¥06-107Mninfmme computing syslems. 24

Muldruw.47 Mathem8!tical mode. 5 Mathem8!ticnl mode1s/modeling. 3. 10-16 MathWorks Inc.， 2s MATLAB.24-44

code snippet. crealing a. 37-40 edilor in. 33-37 日間l-limeuse of. 27. 29-33 bigh-Jeve1 gr冨phicsfunctions of. 48-50 insla11ation of. 26-28 operuωrs in. 30-31 Optimization T，∞1box of. 123-124.

379-391 progrnm.c陀atinga， 40-44

Simp1ex Method using. 120-124 Symbolic Math T，∞1box of. 159-161 usefu1ness of， 26

Matricω: augmenlω.103-104

396 INDEX

MBIIIαs (continue.のrank of， 104-106

Mu且imumproblems， 6 M-fi1ω，27 Minimization problems， 6 Modified Newton me曲。d，252-253Mu1tidisciplinary designs， 6 Mu1tiobjcctive dωigns， 6

n-dimensional Euclidenn spaα:5， 106 Newton-Raphson technique， 206-209， 252 Non1incar programming (NLP) prob1cms，

21-23， 154-2'叩W1a1ytiω1 conditions in， 175-176 cons町aintsin， 157-159， 172-175，

179-191 examp1es of. 194-2∞ firsl-ordcr∞nditions in， 176-178 formu1ation ofproblem， 155-157 graphica1 solutions of， 171-175 Kuhn-Tuckcr conditions for， 192-193，

196-2∞ Lagrangc multiplier mcthod， use of，

180-192 LP prob1ems VS.， 20， 93-94 one variable， functions of， 162-164，

169-170 mωnd・0吋erconditionsin， 178-179 symbolic computotion， 159-161 Tay10r scries for use in solving， 169-171 twoormo陀 varillblcs，functions of，

164-171

Obj民 tivcfunctions， 6 0吋cctorientcd program回目g，24One-dimcnsional problems， 203-225

biscction tccbnique for use wi血，209-210constr瓜ned，204-205dcfinition of problem， 204 golden scction mcthod for use with，

214-217，219-220，223-225 g匂Mα1solution of， 205-206 importance of， 217-219 Newton-Rapbson tcc加岨quefor use with，

206-209 polynomia1 approximBtion with， 211-214 two-point boundary value problem，

220-223 Operators， 30-31

句timalsolu脚色 17Optimization，l-4 Optimization Toolbox， 123-124， 379-391

constraincd optimil.ation using， 389-391 examples using， 385-386 1inear progmmming examplc using，

385-386 optimization parameters， setting， 384-385 qua也温icprogr四 uningcxamp1e using，

386-388 unconstr百inedoptimization using，

388-389 using progmms in， 382-384

OUTPUT program (Optimization Too1box)，

383

Parameほお，d弱 ign，5 Partial derivativ凶， 1ω Pascnl，24 Pntlem Scarch metbod， 234-238 Po1ynol1u叫阻pproximation，211-214， 217 Powell's mctbod， 238-240

a1gori山m，238-239application， 239-240 and conjugacy， 240

Primal problem， 138 Problcm defi副 onandfiロrmu1ation，4-10_

See also Matbematical modelsJ modeling

co目S甘凶nedoptimization problems， 266-267

∞nstr泊ntfunctions in， 7 design functions in， 5-6 dωi伊 par百netersin， 5 and design spaα，9 design vnriab1es in， 4-5 句叫ity∞nstr泊目白i民 7-8global optimil.atiロn，351-357gruphicu1 optimization， 45-48 inequality constraints in， 8 nonlinear problems， 21.，.23 目白目Iin伺 rprogramming (NLP) prob1ems，

155-159 objccuve function in， 6 one-dimcnsional prob1em， 204-205 side∞ns甘aintsin， 8 standard format for， 9-10 unconstmincd optimization problems，

227-230

Prob1em rclaxation， 321-322 Problem solutions， 16-24 Programming， discrcte， 20 針。gramming，intcg白， 20Programming languagω，24 Programming pamdigms， 24 Pseudornndom numbers， 230

Quadratic convergence， 238 Qua命aticpo1ynomials， 238 Qua世aticprogr副加凶ng，386-388

Rundom Walk method， 230-233， 254-255 Redundancy，101 Rosenbrock problem， 253-255 Rota也Igdisk， thrce-dimensional 110w near，

255-258

SA， see Simulated Annenling Script m-fiIes， 27 S回開ha1gori出m，218Search dircction， 22 Search me曲ods，21 Second-order conditions (SOC)， 178-179 ScωIId-urderlllnear variation， 169 Sensitivity an叫ysis，148-151 Sequential dccision problems， 336 Sequentinl Gradient Restoration a1goritbm

(SGRA)， 302-307 nlgoritbm， 305 appli回出n，305-3σ7gradient phnse， 303-304 restoration phase， 304-305

Sequential Linear Progr沼町凶ng(SLP)， 284-289

叫gori山m，285-286application， 286-289

Sequential Qua世atlcProgmmming (SQP)， 289-296

a1gorithm， 294 application， 295-296 and BFGS method， 293-294 multipliers， calculation of， 293 stepsize， ca1culation of， 292-293

Sequential Unconstrnincd Minimization Tcchniques (SUM'η，270-271

SGRA， see Sequentlul Omdicnt Restoration aJgoritbm

Side constraints， 8

Simplcx method， 115-124 application of， 117 -120

INDEX 397

Entering Basic Variable (EBV)， 118 features of， 115-117 Leaving Basic Variub1c (LBV)， 118 MATI-AB， solution using， 120-124 two-phase， 125-130

Simulllted Annealing (SA)， 350， 358-366 a1temativc a1gori血m，359-365basic algorithm of， 358-359 ∞nstraints， hand1ing， 364 in dlscrcte problcms， 365-366

Slack variab1es， 95 SLP， see Sequential Lincar町'OgrammingSOC， see Sccond・ordcrconditions Softw町民 24Solutions， 16-24

acceptoblc， 17 infinite， 17 initial design， 21 optim叫， 17

SQP， see Sequential Quadrntic Programming Stnndard formal， 9-10 Stecpest Dωcentme曲。d，241-244SUMT， J-ee Scquenti凶 Unconstrained

Minimization Tcchniques Symbo1ic Math T，ロolbox(MATLAB)，

159-161

Taylor scric唱， 169-171 Tol町ance，209 TPBVP， sec Two-point boundary yalue

problem Trans戸市山nproblcm example， 124-130 Twcトpointboundary va1ue problcm

σ'PBVP)，220-223

Unconstrained oplimizlltion problems， 227-262， 388-389

Bczicr paramctriccurves， fittlng， 258-262

Broydon-F1etcher-Go1dfarl:トShannomethod for solving， 249-251

Co吋ugateGradient mctbod for solving， 244-246

Davidon-F1etchcr-Powcll melhod for solving，24ι249

definition of prob1em， 227-230 generic a1goritbm for， 229-230

396 INDEX

Malri臨 (continued)

rankof，l04-106 M脱出umproblcms. 6

M-filω，27

Minimization problcms， 6 ModifiωNewton method， 252-253 Mullidisciplinnry designs， 6 Multiobjeclive designs， 6

n-dimensional Euclidean spaces， 106 NewlOn-Rapbso目tech副quc，206-2ω，252 Nonlinear programming (NLP) problem~，

21-23， 154-200 analytical condItions in， 175-176 constrllInts in， 157-159， 172-175，

179-191

cxampl隠。ι194-200

目指1-0凶ercondilions in， 176-178 伽 mulationof problern， 155-157

graphicalsolutions of， 171-175 Kuhn・Tuckerconditions for， 192-193，

19ふ2∞Lagmnge mullIplier melhod， use of，

180-192

LP problems vs.， 20， 93-94 one varinble. funclIons of. 162-1 64.

169-170

second-order condIlions in， 178-179 syrnbolic computa凶on，159-161 Tl1ylorseri鎚 foruse in solving. 169-171 two or moro varIables， funclions of，

164-171

Objective functions， 6

0尚氏loricn凶 programming，24

One-dImensIonal problcms， 203-225 biscction technique for use wIth， 209-210 COI首位画ned，204-205definition of problcm， 204 gold叩 scctionmethod for use with.

214-217，219-220，223-225 grnphical solution of， 205 -206 importance of， 217-219 NcwlOn-Raphson lechnique for use with，

206-209 polynomial approximation Wilh， 211-214

two-poInt boundnry valuc problem， 220-223

Opemlo路， 30-31

Optimal solutions， 17 句伽凶ion，l-4Optimization T，ωlbox， 123-124，379-391

conslmIncd optImiznlion using， 389-391 examples using， 385-386 linear prograrmning ex田npleusIng，

385-386

optimizalion p町耳me股rs，seuing， 384・・385qua企冨.ticprogramming cxample using，

386-388

un∞田町司inedoptimization us泊E・388-389

using programs In， 382-384 0げrpUTprogmm (Optimization T，∞lbox).

383

Parameters， design， 5

Panial derivatives， 164 Pascal，24 PattcmS叫 rchmcthod， 234-238 Polynomial opproximation， 211-214. 217 Powell's me山od，238-240

a1gori白m，238-239app1同町n.239-240and conjugacy， 240

Primal problem， 138 Problem definition and formu1ation， 4-10.

See also Malhematical mode1s1 modeling

cons町話回doplimizatioD problems， 266-267

∞nsuaint functions in， 7 design functions in， 5-6 design paramete四 in，5and design space， 9

design vnriablcs in， 4-5 equality consuaints in， 7-8 globalロ，pt同比ation，351-357 graphical oplimization， 45-48

inequa1ity∞nsuaints in， 8 nonlinear problems， 21ヲ23nonlinear programming (NLP) problems，

155-159 0同氏tivefunctioD in， 6 one-dimensional problem， 204-205 side constmints in， 8

standord formnt for， 9-10

uncons回 incdoptimization problems， 227-230

Problemrel以 ation，321-322Problem solutions， 16ー24Progmmming， disc陀 te，20Programming， integcr， 20

PrognunoUnglanguages，24 PrognunoUng paradigms， 24 Pseudorandom numbers， 230

Quadratic convergenω，238 Quadratic polynomials， 238 Quadmtic programming， 386-388

Random Wa1k method， 230-233， 254-255 R吋 undancy，101 R05cnbrock problem， 253-255

Rotating disk， thr官c-dimensionalflow near，お5-258

SA， .rtt Simulated Annealing Script m-filcs， 27 Search a1gorithm， 218 S側同hdi悶 tion，22S阻 rchmethods， 21

Second-ordcr conditions (SOC)， 178-179 Second-or枇r/linearvarintion， 169 Sensitivilyannlysis， 148-151 S句uentiald民iSIODproblems， 336

Sequcntial Gradient Restoration a1gorithm (SGRA)， 302-307

a1gori白m，305application， 305-307 gradient phωe，303-304 restoratiOD phase. 304-305

Sequential Linear Programming (SLP)， 284-289

algorithm， 285-286 app1ication， 286-289

S巴quentialQuadratic Programming (SQP)， 289-296

algori山m.294applicntion， 295-296

andBFGSme山od，293-294 mu1tipliers，ωI印刷ionof，293

stepsize， calculation of， 292-293 Sequential Unconsuaincd恥鉛IImi7.otion

Techniques (SUMT>， 270-271 SGRA， soe ScqucntiaJ Gradienl Res[oration

n1gロrithmSidc∞ns凶 nts，8

Simplex method， 115-124 叩plicationof， 117-120

INDEX 397

Entering Basic Variable (EBV)， 118 features of， 115-117 Leaving 8asic Variable (LBV)， 118 MATlAB， solulion using， 120-124 two-phasc， 125-130

Simul回edAnnealing (SA)， 350， 358-366 a1temative algorithm， 359-365 basic a1gorithm of， 358-359 ∞n5U司in也， h邸ldJing，364

in discrete problems， 365-366 Slack variables， 95

SLP， see Sequenlial Linear Programming SOC， sl!e Sccond-order conditions So食ware，24Solutions， 16-24

a∞eptab1e，I7

infinite， 17 inilial design， 21 optimal，17

SQP， see Sequential Quudrntic Prograrnming Standard fonnnt， 9-10 St白戸stO飽αntmethod， 241-244 SUMT，seeS句uentialUnconstmined

MlnImization Techniques Sym加licMathT，∞lbox(MATlAB)，

159-161

Taylor scries， 169-171 To1erunce， 209

TPBVP，s館Two・pointbound町yva1ue problem

Transpon瓜ionprob1em cx組司p1e，124-130 Twcトpointboundnry value problem

σ'PBVP)，220-223

Un∞n5U百inedoptimization problems， 227-262，388-389

8ezicr pnrametric cu円 ω，fitting， 258-262

8roydon-Fletcher-GoldfartトShanno

method for 50lving， 249-251 Conjugate Gradient melhod for 801ving，

244-246 Oavidon-F1clcher-PowclI melhod for

solving，24ι249 deflDItion of problem， 227-230 generic a1gorithm for， 229-230

E留置箇固

398 INDEX

Uncorullrained oplimiz.瓜ionproblems

(conル拙eのg百dient-based町lethodsfor solving，

241-251 modified NeWlOn method for solving，

252-253 neces姐ryIlDd sufficient conditions for，

228-229 nongradient melhods for solving，

230ー240Pattem S朗氏hmethod for solving，

234-238 PowcU's melhod for solving， 238-240 random wa1k a1gorilhm for， 230-233 Rosenbrock problem， 253-255 rotating disk， thrce-dimensional sow

目国r，255-おS

second-order melhods for solving， 251ーお3

Sleepcst Dcscent mclhod for solving，

241-244

Unique optirna1 solutions. 17 Unitv出 ors，106-107

Varillble Metric Methods (VMM)， 246 Varinbles:

nrtificiHl， 98 continuous， 20 desi伊，4-5discrele， 20

integ民 20slack，95

Vecωr(s): design， 5 unit， lO岳ー107

VMM (Variable Metric Melhods)， 246

Zero-Oneln崎町Programrning(ZIP)， 343-348

Zero-order methods， 230 ZIP， see Zero-One lnleger Programming

applied optimization with matlab programming

Documents