general nonlinear programming (nlp) softwarecs777/presentations/nlp-software.pdf · general...
TRANSCRIPT
![Page 1: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/1.jpg)
General Nonlinear Programming (NLP) Software
CAS 737 / CES 735
Kristin DaviesHamid Ghaffari
Alberto Olvera-SalazarVoicu Chis
January 12, 2006
![Page 2: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/2.jpg)
OutlineIntro to NLPExamination of:
IPOPT PENNON CONOPT LOQO KNITRO
Comparison of Computational ResultsConclusions
![Page 3: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/3.jpg)
Intro to NLPThe general problem:
{ }{ }
.
,,,
,...,1,0)(,...,1,0)(..
)(min
Condefinedfunctionsare
ghfandsetcertainaisCxwhereCx
mJjxgpIixhts
xf
jinn
j
i
ℜ⊆ℜ∈
∈
=∈≤=∈=
Either the objective function or some of the constraints may be nonlinear
(NLP)
![Page 4: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/4.jpg)
Intro to NLP (cont’d…)Recall:
The feasible region of any LP is a convex setif the LP has an optimal solution, there is an extreme point of the feasible set that is optimal
However:even if the feasible region of an NLP is a convex set, the optimal solution might not be an extreme point of the feasible region
![Page 5: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/5.jpg)
Intro to NLP (cont’d…)Some Major approaches for NLP
Interior Point MethodsUse a log-barrier function
Penalty and Augmented Lagrange MethodsUse the idea of penalty to transform a constrained problem into a sequence of unconstrained problems.
Generalized reduced gradient (GRG)Use a basic Descent algorithm.
Successive quadratic programming (SQP)Solves a quadratic approximation at every iteration.
![Page 6: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/6.jpg)
Summary of NLP Solvers
PENNON
AugmentedLagrangian Methods
KNITRO (TR)IPOPT, LOQO(line search)
Interior PointMethods
CONOPT
Reduced GradientMethods
NLP
![Page 7: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/7.jpg)
IPOPT SOLVER (Interior Point OPTimizer)
CreatorsAndreas Wachter and L.T. Biegler at CMU (~2002)
AimsSolver for Large-Scale Nonlinear Optimization problems
ApplicationsGeneral Nonlinear optimizationProcess Engineering, DAE/PDE Systems, Process Design and Operations, Nonlinear Model Predictive control, Design Under Uncertainty
![Page 8: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/8.jpg)
IPOPT SOLVER (Interior Point OPTimizer)
Input FormatCan be linked to Fortran and C code MATLAB and AMPL.
Language / OSFortran 77, C++ (Recent Version IPOPT 3.x)Linux/UNIX platforms and Windows
Commercial/FreeReleased as open source code under the Common Public License (CPL). It is available from the COIN-OR repository
![Page 9: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/9.jpg)
IPOPT SOLVER (Interior Point OPTimizer)
Key ClaimsGlobal Convergence by using a Line Search.
Find a KKT pointPoint that Minimizes Infeasibility (locally)
Exploits Exact Second DerivativesAMPL (automatic differentiation)If not Available use QN approx (BFGS)Sparsity of the KKT matrix.
IPOPT has a version to solve problems with MPEC Constraints. (IPOPT-C)
![Page 10: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/10.jpg)
IPOPT SOLVER (Interior Point OPTimizer)
AlgorithmInterior Point method with a novel line search filter.Optimization Problem
00)(..
)(min
≥=
ℜ∈
xxcts
xfnx
0)(..
)log()()(min )(
=
−= ∑ℜ∈
xcts
xxfxi
il
xln
μϕμ
The bounds are replaced by a logarithmic Barrier term. The method solves a sequence of barrier problems for decreasing values of μl
![Page 11: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/11.jpg)
IPOPT SOLVER (Interior Point OPTimizer)
Algorithm(For a fixed value of )
Solve the Barrier ProblemSearch Direction (Primal-Dual IP)
Use a Newton method to solve the primal dual equations.Hessian Approximation (BFGS update)
Line Search (Filter Method)Feasibility Restoration Phase
lμ
![Page 12: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/12.jpg)
IPOPT SOLVER (Interior Point OPTimizer)
Optimization Problem
00)(..
)(min
≥=
ℜ∈
xxcts
xfnx
0)(..
)log()()(min )(
=
−= ∑ℜ∈
xcts
xxfxi
il
xln
μϕμ
lμ
Outer LoopOuter Loop
The bounds are replaced by a logarithmic Barrier term.
The method solves a sequence of barrier problems for decreasing values of
![Page 13: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/13.jpg)
IPOPT SOLVER (Interior Point OPTimizer)
Algorithm (For a fixed value of )Solve the Barrier Problem
Search Direction (Primal-Dual IP)Use a Newton method to solve the primal dual equationsHessian Approximation (BFGS update)
lμ
![Page 14: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/14.jpg)
IPOPT SOLVER (Interior Point OPTimizer)
InnerLoopInnerLoop
0)(0
0)()(
==−
=−∇+∇
xceXVe
vxcxfμ
λ
Optimality conditionsOptimality conditions
eXvVariablesDual
1−= μ
0)(..
)log()()(min )(
=
−= ∑ℜ∈
xcts
xxfxi
il
xln
μϕμ Barrier Barrier NLPNLP
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−
−∇+∇−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡∇
−∇
eeVXxc
vxcxf
ddd
VXxc
IxcH
kk
k
kkT
kk
vk
k
xk
kk
Tk
kk
μ
λλ )(
)()(
000)(
)(
),(2kkxxk xLH λ∇=
At a Newton's iteration At a Newton's iteration ((xxkk,,λλkk,v,vkk))
⎥⎦
⎤⎢⎣
⎡ −∇+∇−=⎥
⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡−∇∇+Σ+
)()()(
)()(
k
kkT
kk
k
xk
cT
k
kHkk
xcvxcx
dd
IxcxcIH λϕδ
δ μλ
At a Newton's iteration At a Newton's iteration ((xxkk,,λλkk,v,vkk))
Algorithm Core: Solution of this Linear systemAlgorithm Core: Solution of this Linear system
![Page 15: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/15.jpg)
IPOPT SOLVER (Interior Point OPTimizer)
Algorithm (For a fixed value of )Line Search (Filter Method)
A trial point is accepted if improves feasibility or if improves the barrier function
Assumes Newton directions are “Good”especially when using Exact 2nd Derivatives
lμ
vkkkk
xkkkk
dvv
dxx
α
α
+=
+=
+
+
1
1
[ ] [ ]
[ ] [ ]kkk
kkk
xxor
xcxc
ϕαϕ
α
≤
≤
)(
)(If
![Page 16: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/16.jpg)
IPOPT SOLVER (Interior Point OPTimizer)
Line Search - Feasibility Restoration Phase
When a new trial point does not provides sufficient improvement.
0..
)(min 2
2
≥ℜ∈
xts
xcnx
Restore FeasibilityMinimize constraint
violation
00)(..
min 2
2
≥=
−ℜ∈
xxcts
xx kx n
Force Unique SolutionFind closest feasible point.
Add Penalty function
![Page 17: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/17.jpg)
The complexity of the problem increases when complementarity conditions are introduced from:
miywywx
ywxcst
ywxf
ii
ywx mmn
K1,00,,
0),,(.
),,(min
)()(
,,
==
≥=
ℜ∈ℜ∈ℜ∈
( ) ( ) ( )( )
miywywxc
st
ywx
ywxf
ii
m
iim
iin
ii
ywx mmn
K1,00),,(
.
lnlnln
),,(min
)()(
1)(
1)(
1)(
,,
==
=
++− ∑∑∑ ===
ℜ∈ℜ∈ℜ∈
μ
δμ≤)()( ii yw
•The interior Point method for NLPs has been extended to handle complementarity problems. (Raghunathan et al. 2003).
0)()( =ii yw is relaxed as 0)(
)()()(
≥
=+i
iii
ssyw δμ
IPOPT SOLVER (Interior Point OPTimizer)
![Page 18: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/18.jpg)
IPOPT SOLVER (Interior Point OPTimizer)
AdditionalIPOPT 3x. Is now programmed in C++.Is the primary NLP Solver in an undergoing project for MINLP with IBM.
ReferencesIpopt homepage:
http://www.coin-or.org/Ipopt/ipopt-fortran.htmlA. Wächter and L. T. Biegler, On the Implementation of a Primal-Dual Interior Point Filter Line Search Algorithm for Large-Scale Nonlinear Programming, Research Report, IBM T. J. Watson Research Center, Yorktown, USA, (March 2004 - accepted for publication in Mathematical Programming)
![Page 19: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/19.jpg)
PENNON (PENalty method for NONlinear& semidefinite programming)
CreatorsMichal Kocvara & Michael Stingl (~2001)
AimsNLP, Semidefinite Programming (SDP), Linear & Bilinear Matrix Inequalities (LMI & BMI), Second Order Conic Programming (SOCP)
ApplicationsGeneral purpose nonlinear optimization, systems of equations, control theory, economics & finance, structural optimization, engineering
![Page 20: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/20.jpg)
SDP (SemiDefinite Programming)
Minimization of a linear function subject to the constraint that an affine combination of symmetric matrices is positive semidefinite
∑=
+=
≥m
iii
T
FxFxFwhere
xFtsxc
10)(
0)(..min
),...,(matricessymmetric1 0 mFFm +
Linear Matrix Inequality (LMI)defines a convex constraint on x
![Page 21: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/21.jpg)
SDP (SemiDefinite Programming)
-always an optimal point on the boundary
-boundary consists of piecewise algebraic surfaces
![Page 22: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/22.jpg)
SOCP (Second-Order Conic Programming)
Minimization of a linear function subject to a second-order cone constraint
iTiii
T
dxcbxAtsxc
+≤+..min
⎭⎬⎫
⎩⎨⎧
≤∈∈⎥⎦
⎤⎢⎣
⎡= − tuRtRu
tu
C kk ,,1
Called a second-order cone constraint since the unit second-order cone of dimension k is defined as:
Which is called thequadratic, ice-cream,or Lorentz cone
![Page 23: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/23.jpg)
PENNON (PENalty method for NONlinear& semidefinite programming)
Input FormatMATLAB function, routine called from C or Fortran, stand-alone program with AMPL
LanguageFortran 77
Commercial/FreeVariety of licenses ranging from
Academic – single user ($460 CDN) to Commercial – company ($40,500 CDN)
![Page 24: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/24.jpg)
PENNON (PENalty method for NONlinear& semidefinite programming)
Key Claims1st available code for combo NLP, LMI, & BMI constraintsAimed at (very) large-scale problemsEfficient treatment of different sparsitypatterns in problem dataRobust with respect to feasibility of initial guessParticularly efficient for large convex problems
![Page 25: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/25.jpg)
PENNON (PENalty method for NONlinear& semidefinite programming)
AlgorithmGeneralized version of the Augmented Langrangian method (originally by Ben-Tal & Zibulevsky)
( ) 0/)(..)(min
≤iigi pxgptsxfϕ
Augmented Lagrangian
( )∑=
+=gm
iiigii pxgpuxfpuxF
1/)()(),,( ϕ
multiplierLagrange
functionpenaltyparameterpenalty0
sconstraintinequalityof#
,...,1
=
==>
=Augmented Problem =
i
g
i
g
g
u
p
m
mi
ϕ
![Page 26: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/26.jpg)
PENNON (PENalty method for NONlinear& semidefinite programming)
The AlgorithmConsider only inequality constraints from (NLP)
Based on choice of a penalty function, φg, that penalizes the inequality constraints
Penalty function must satisfy multiple properties such that the original (NLP) has the same solution as the following “augmented” problem:
( )0
,...1,0/)(..,)(min
>
=≤ℜ∈
i
giigi
pwith
mipxgptsxxf
ϕ (NLPφ)
[3] Kocvara & Stingl
![Page 27: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/27.jpg)
PENNON (PENalty method for NONlinear& semidefinite programming)
The Algorithm (Cont’d…)The Lagrangian of (NLPφ) can be viewed as a (generalized) augmented Lagrangian of (NLP):
( )∑=
+=gm
iiigii pxgpuxfpuxF
1/)()(),,( ϕ
Lagrange multiplierPenalty parameter
Penalty function
Inequality constraint
[3] Kocvara & Stingl
![Page 28: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/28.jpg)
PENNON (PENalty method for NONlinear& semidefinite programming)
The Algorithm STEPS.,...,1,0.)0( 111
gii mipLetgivenbeuandxLet =>
.,...2,1
satisfiediscriteriumstoppingauntilrepeatkFor =
[3] Kocvara & Stingl
( )( )( )
gki
ki
gki
kig
ki
ki
kkkx
k
mippiii
mipxguuii
KpuxFthatsuchxFindi
,...,1,)(
,...,1,/)(
,,)(
1
1'1
11
=<
==
≤∇
+
++
++
ϕ
![Page 29: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/29.jpg)
PENNON (PENalty method for NONlinear& semidefinite programming)
The Algorithm STEPS
.,...,1,0.)0( 111gii mipLetgivenbeuandxLet =>
InitializationCan start with an arbitrary primal variable , therefore, chooseCalculate initial multiplier valuesInitial p= , typically between 10 - 10000
1iu
01 =xx
[3] Kocvara & Stingl
0π
![Page 30: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/30.jpg)
PENNON (PENalty method for NONlinear& semidefinite programming)
The Algorithm STEPS
( ) KpuxFthatsuchxFindi kkkx
k ≤∇ ++ ,,)( 11
(Approximate) Unconstrained MinimizationPerformed either by Newton with Line Search, or by Trust RegionStopping criteria:
[3] Kocvara & Stingl
( )( ) ( )( )( ) ( ) 11 ,,,,
/,,
1.0,,
1
2
1'
2
1
2
1
−−∇⋅≤∇
−⋅≤∇
=≤∇
+
++
+
H
kkkxH
kkkx
ki
kig
ki
ki
kkkx
kkkx
puxFpuxF
orpxguupuxF
orpuxF
α
ψα
αα
1)'(' −= ϕψ),,(minarg1 kk
x
k puxFx =+
![Page 31: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/31.jpg)
PENNON (PENalty method for NONlinear& semidefinite programming)
The Algorithm STEPS
5.0,1 typically≤μ
( )( ) gki
kig
ki
ki mipxguuii ,...,1,/)( 1'1 == ++ ϕ
Update of MultipliersRestricted in order to satisfy:
with a positive
If left-side violated, let If right side violate, let
μμ 11
<<+
ki
ki
uu
μ=newiu
μ/1=newiu
[3] Kocvara & Stingl
![Page 32: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/32.jpg)
PENNON (PENalty method for NONlinear& semidefinite programming)
The Algorithm STEPS
gki
ki mippiii ,...,1,)( 1 =<+
Update of Penalty ParameterNo update during first 3 iterationsAfterwards, updated by a constant factor dependent on initial penalty parameterPenalty update is stopped if peps (10-6) is reached
[3] Kocvara & Stingl
![Page 33: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/33.jpg)
PENNON (PENalty method for NONlinear& semidefinite programming)
The Algorithm
Choice of Penalty FunctionMost efficient penalty function for convex NLP is the quadratic-logarithmic function:
[4] Ben-Tal & Zibulevsky
holdpropertiesthatsocrtcctc
andrwherertctctct
i
g
6,...,1)log(
)1,1()(
654
322
21
1
=<+−
−∈≥++=ϕ
![Page 34: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/34.jpg)
PENNON (PENalty method for NONlinear& semidefinite programming)
The Algorithm
Overall Stopping Criteria
[3] Kocvara & Stingl
7
1
10
)(1
)()(
)(1
),,()(
−
−
=
<+
−<
+
−
ε
εε
where
xf
xfxfor
xf
puxFxfk
kk
k
kkk
![Page 35: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/35.jpg)
PENNON (PENalty method for NONlinear& semidefinite programming)
Assumptions / WarningsMore tuning for nonconvex problems is still requiredSlower at solving linear SDP problems since algorithm is generalized
![Page 36: General Nonlinear Programming (NLP) Softwarecs777/presentations/NLP-Software.pdf · General Nonlinear Programming (NLP) Software CAS 737 / CES 735 Kristin Davies Hamid Ghaffari Alberto](https://reader030.vdocument.in/reader030/viewer/2022040114/5e4c85cde8869476285db3b3/html5/thumbnails/36.jpg)
PENNON (PENalty method for NONlinear& semidefinite programming)
ReferencesKocvara, Michal & Michael Stingl. PENNON: A Code for Convex and Semidefinite Programming. Optimization Methods and Software, 8(3):317-333, 2003.Kocvara, Michal & Michael Stingl. PENNON-AMPL User’s Guide. www.penopt.com . August 2003.Ben-Tal, Aharon & Michael Zibulevsky. Penalty/Barrier Multiplier Methods for Convex Programming Problems. Siam J. Optim., 7(2):347-366, 1997.Pennon Homepage. www.penopt.com/pennon.html Available online January 2007.