numerical nonlinear optimization with worhp · numerical nonlinear optimization with worhp christof...
TRANSCRIPT
![Page 1: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/1.jpg)
Centre for Industrial Mathematics Centre for Industrial Mathematics
Numerical Nonlinear Optimization
with WORHP
Christof Büskens
London, 8.9.2011
Optimierung &
Optimale Steuerung
![Page 2: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/2.jpg)
Centre for Industrial Mathematics
Outline
Nonlinear Optimization
WORHP
Concept
Ideas
Features
Results
Optimization &
Optimal Control
![Page 3: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/3.jpg)
Centre for Industrial Mathematics
WG Optimisation and Optimal Control (2004)
16 + 4 scientists + additional staff
Industrial & scientific research
![Page 4: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/4.jpg)
Centre for Industrial Mathematics
Working Areas
Parametric
Sensitivity
Analysis
Modeling &
Simulation
Real Time
Optimization &
Optimal Control
Identification
Optimization
Optimal
Control
Feedback
Control
Mathematics
Theory
Scientific Computation
![Page 5: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/5.jpg)
Centre for Industrial Mathematics
Sparse NLP Solver
WORHP
We Optimize Really Huge Problems
> 1.000.000.000 variables
> 2.000.000.000 constraints
![Page 6: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/6.jpg)
Centre for Industrial Mathematics
WORHP, just another NLP solver?
<1997
SNOPT
2000
IPFILTER
2001
KNITRO
2002
IPOPT
2010
WORHP
History of sparse NLP solvers:
Can WORHP be competative?
![Page 7: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/7.jpg)
Centre for Industrial Mathematics
User, Market & Scientific Requirements In contrast to most established and „grown”
NLP solvers, WORHP has undergone
extensive design on the drawing board before
its implementation was started, making use of
-user requirements,
-current architectures,
-computational standards and compilers
to construct a modern NLP solver for large-
scale nonlinear optimisation.
224 Solvers
28 for nonlinear constraints
19 dense & sparse
Programming Language
C / C++
3Fortran / C
9Fortran
16
Commercialization / License Type
Company; 9
GPL type; 3
Direct; 14
Distribution by Country
AT; 2
DK; 1
CH; 1
PT; 1
BE; 1
FI; 1
RU; 1
GR; 1
BR; 1
DE
5
UK
5
AU; 1
US
13
EU/ESA
16
Algorithms
12
8
5
1 1 1 1
Sequential Quadratic Programming
Primal-Dual Interior Point
Generalized Reduced Gradient
Successive Quadratic Programming
Benders Decomposition
Levenberg Marquardt
Coordinate Search
SQP
IP
GR
Optimization Classes
4330
36
4
47
2
1
1014
11
6
20
LP-problem, mixed integer, stochastic
QP-problem, mixed integer
Semidefinite and 2nd-order cone prog.
Geometric programming
Non Linear programming
Minimization of nonsmooth functions
Semi-infinite programming
Mixed integer nonlinear programming
Network constraints
Special/constraint solvers
Dynamic programming
Control problems
![Page 8: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/8.jpg)
Centre for Industrial Mathematics
The WORHP Team in Bremen
Prof. Dr.
Christof Büskens
Project leader Bremen
Dipl. math.
Tim Nikolayzik
NLP-Correction
Methods
Dennis Wassel, MPhil
“Sentinel” - Project,
associated with NLP
SADCO Project
Sonja Rauski
Hessian-Approximations
![Page 9: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/9.jpg)
Centre for Industrial Mathematics
n: large m: large nonlinear
iterative solution
Nonlinear Optimization
![Page 10: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/10.jpg)
Centre for Industrial Mathematics
iteration number
iterative solution
Nonlinear Optimization
![Page 11: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/11.jpg)
Centre for Industrial Mathematics
line search or filter search direction merit function
step size
Nonlinear Optimization
![Page 12: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/12.jpg)
Centre for Industrial Mathematics
1st derivatives
2nd derivatives or approx. (BFGS)
globalisation
relaxation
sparse structure
General Idea/Problems
![Page 13: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/13.jpg)
Centre for Industrial Mathematics
globalisation
relaxation
linear algebra
General Idea/Problems
![Page 14: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/14.jpg)
Centre for Industrial Mathematics
WORHP (SQP/IP)
computational action
mathematical action
![Page 15: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/15.jpg)
Centre for Industrial Mathematics
Finite Differences
are expensive
are inexact
are only interesting for „black-box“ problems
Really?
![Page 16: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/16.jpg)
Centre for Industrial Mathematics
Group Approach by Graph Coloring
Assuming the Jacobian to have the following structure:
Usual approach:
Group approach:
(Extension for second derivatives by pair groups)
Numerical example: (Rayleigh optimal control problem)
Usual: 100.000 evaluations
Group: 6 evaluations (NP-hard)
![Page 17: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/17.jpg)
Centre for Industrial Mathematics
Complex Numerical Differentiation [Martins, Kroo, Alonso]
Classical Approach:
Consider
Cauchy Riemann„s equation:
Hence:
No Cancellation Error!
![Page 18: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/18.jpg)
Centre for Industrial Mathematics
SBFGS (Sparse-BFGS)
Consider
SBFGS considers this sparsity structure
SBFGS performs a BFGS update on the three blocks
Problem: Intersections!
Solution: Convexity Shifts
![Page 19: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/19.jpg)
Centre for Industrial Mathematics
SBFGS
Theorem: (Superlinear Convergence) [Kalmbach, B.]
Let and appropriate functions,
s.t. and
for all and proper, symmetric
and positive definite start matrices.
Then converges superlinearly towards .
Proof: Segmentation, rang 2M update, convexity shift of kernel,
+[Griewank, Toint]: Local Convergence Analysis for Partitioned Quasi-Newton Updates, 1982.
![Page 20: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/20.jpg)
Centre for Industrial Mathematics
Interfaces Fortran
Traditional
Basic-Feature
Full-Feature
C/C++
Traditional
Basic-Feature
Full-Feature
AMPL
MATLAB/SIMULINK
Plattforms:
Linux/Unix
Windows
Mac OS
![Page 21: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/21.jpg)
Centre for Industrial Mathematics
CUTEr 920 problems (academical & real life)
Small and dense
Large and sparse
Implemented in AMPL
Solvers used for validation:
SNOPT KNITRO IPOPT WORHP
[<1997] [2001] [2002] [2010]
![Page 22: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/22.jpg)
Centre for Industrial Mathematics
CUTEr(920)
SNOPT
7.2-8
Problems solved 827
Optimal level 810
Acceptable level 17
Not solved 93
Percentage 89,89%
Time 79569 s
KNITRO
7.0.0
887
882
5
33
96,41%
32792 s
IPOPT
3.9.2
877
869
8
43
95,33%
27056 s
WORHP
1.0
918
911
7
2
99,79%
5060 s
![Page 23: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/23.jpg)
Centre for Industrial Mathematics
CUTEr
![Page 24: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/24.jpg)
Centre for Industrial Mathematics
CUTEr
![Page 25: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/25.jpg)
Centre for Industrial Mathematics
CUTEr
![Page 26: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/26.jpg)
Centre for Industrial Mathematics
COPS 3.0 68 problems (applications)
Sparse
Midsized and Large
Implemented in AMPL
Solvers used for validation:
SNOPT KNITRO IPOPT WORHP
![Page 27: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/27.jpg)
Centre for Industrial Mathematics
COPS 3.0(68)
SNOPT
7.2-8
Problems solved 63
Optimal level 60
Acceptable level 3
Not solved 5
Percentage 94,12%
Time 8858 s
KNITRO
7.0.0
64
64
0
4
94,12%
6352 s
IPOPT
3.9.2
68
68
0
0
100%
5682 s
WORHP
1.0
68
67
1
0
100%
1463 s
![Page 28: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/28.jpg)
Centre for Industrial Mathematics
COPS 3.0
![Page 29: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/29.jpg)
Centre for Industrial Mathematics
New large-scale NLP solver WORHP
> 1,000,000,000 variables
> 2,000,000,000 constraints
Derivatives and SBFGS
Several interfaces
Robust
Real life application
Conclusion
![Page 30: Numerical Nonlinear Optimization with WORHP · Numerical Nonlinear Optimization with WORHP Christof Büskens London, 8.9.2011 Optimierung & Optimale Steuerung. Centre for Industrial](https://reader035.vdocument.in/reader035/viewer/2022062302/5f039a5d7e708231d409ddc4/html5/thumbnails/30.jpg)
Centre for Industrial Mathematics
Thank you!
Please visit: www.worhp.de