non-uniform covering method for global optmization and...
TRANSCRIPT
Non-uniform covering method for global optmization and
approximation
Yu. G. Evtushenko, M.A. PosypkinDorodnicyn Computing Centre, FRC CSC RAS
«Optimization at Work», 14 April 2018
МФТИ
Non-Uniform Covering Method1. Initially proposed for global optimization problems in 1971
Ю. Г. Евтушенко, “Численный метод поиска глобального экстремума функций
(перебор на неравномерной сетке)”, Ж. вычисл. матем. и матем. физ., 11:6 (1971),
1390–1403
2. Extened to multiobjective optimization in 1986
Евтушенко Ю. Г., Потапов М. А. «Методы численного решения
многокритериальных задач.» ДАН СССР, Т. 291, N 1, 1986, С. 25-39.
3. Extended to mathematical programming problems in 1992Yu. G. Evtushenko, M. A. Potapov, V. V. Korotkikh.
Numerical methods for global optimization.
In "Recent advances in global optimization”, Princeton University Press, pp. 274-297.
1992.
4. Parallel implementation 2007Ю. Г. Евтушенко, В. У. Малкова, А. А. Станевичюс. Распараллеливание процесса
поиска глобального экстремума. Автоматика и телемеханика, № 5. С. 46-58, 2007.
Non-Uniform Covering Method
1. Advanced techniques for mathematical programming problems 2011Evtushenko Y., Posypkin M. A deterministic approach to global box-constrained
optimization //Optimization Letters. – 2013. – Т. 7. – №. 4. – С. 819-829.
2. Advanced techniques for multiobjective optimizationEvtushenko Y. G., Posypkin M. A. A deterministic algorithm for global multi-objective
optimization //Optimization Methods and Software. – 2014. – Т. 29. – №. 5. – С. 1005-
1019.
3. High-performance computingEvtushenko Y., Posypkin M., Sigal I. A framework for parallel large-scale global
optimization //Computer Science-Research and Development. – 2009. – Т. 23. – №. 3-4.
– С. 211-215.
Evtushenko, Y., Golubeva, Y., Orlov, Y., Posypkin, M.Using simulation for performance
analysis and visualization of parallel Branch-and-Bound methods //Russian
Supercomputing Days. – Springer, Cham, 2016. – С. 356-368.
4. ApproximationEvtushenko Y. G., Posypkin M. A. Effective hull of a set and its approximation //Doklady
Mathematics. – Pleiades Publishing, 2014. – Т. 90. – №. 3. – С. 791-794.
Evtushenko, Y., Posypkin, M., Rybak, L., Turkin, A.. Approximating a solution set of
nonlinear inequalities //Journal of Global Optimization. – 2017. – С. 1-17.
GLOBAL OPTIMIZATION PROBLEM
STATEMENT
Solution set
- solution set
*X X X
* *( ) min ( )
x X
f f x glob f x
* * * *: ( ) X x X f x f
* *: ( ) X x X f f x f
* ( ) f f x
LEBESGUE SET
Xx
( ( ), , ) : ( ) L f x X x X f x
0( ) f x
0x
( ( ), , )L f x X( )f x
define,arbitraryFor nRXR
RECORD POINTS AND VALUES
1 2
1 2
1 1
, ,...,
( ), ( ),..., ( )
min( , ( ))
i
i
i i i
x x x X
f x f x f x
r r f x
Current record value ir
Current record point : ( )
1
j j ix f x r
j i
LOWER BOUNDS
Minorant
Objective function
7
( )if x
1( ) rf x
( )f x
ix] [[]
2( ) rf x
1 2( ) ( )r rf x f x
)()(),()( xxfxfxii
x
NON-UNIFORM COVERING METHOD
[
a
]
b
)(xf
)(1 x
)(2 x
)( jxf
)( ixf
)()(
then),()(
,...,1,where
,))(,),((If
*
1
1
xfxf
xfxf
kiXX
xfXLX
k
k
i
k
i
iii
axfAxAL )(:),),((
Objective
function
Minorants
xi и xj coordinates of atomsn - number of atoms
2
1
1 1
( ) 1 1i j
n nX X
i j i
F e
GLOBAL OPTIMIZATION
PROBLEM EXAMPLE
Morse atomic cluster
energy function
x56 x55
11
Efficient estimating of functions
Lets Hessian spectrum Xi is enclosed in [ki, Ki], then
2
2),()()( i
iiixi xx
kxxxfxfxf
2
2),()()( i
iiixi xx
Kxxxfxfxf
],[)( ii KkH
iiii LKkL
Spectrum bounding
где
Accounting optimality conditions
2
2)()( i
ii xx
Kxfxf
2. The following underestimation is valid on a set of optimal solutions
1. Underestimating on the set of optimal solutions
if
Reducing the search space
Computational experiments
O0
O1
O2
O3
O3 +
BR BARON
LG LINDOGLOBAL
iii xxlxfxf )()(
2
2),()()( i
iiixi xx
Lxxxfxfxf
2
2),()()( i
iiixi xx
kxxxfxfxf
2
2)()( i
ii xx
Kxfxf
2
2)()( i
ii xx
Kxfxf + reduction rules
Computational experiments
GLOBAL MULTIOBJECTIVE
OPTIMIZATION
Problem statement
)(F - continuous function
X - compact set
MULTIOBJECTIVE OPTIMIZATION
nRmR
feasible set
attainable set
)1(x
)2(x
)1(f
)2(f
FY
X
)(XFY
Parameter’s space Objective’s (Criteria’s) space
USEFUL NOTATIONS
yuRuySW m :)(
( ) ( ) 1,...,i iu y u y for i m 1.
2.
yuRuyNE m :)(3.y
)(ySW
)(yNE
(1)y
(2)y
y dominates (better than) each point from SW(y),
each point from NE(y) dominates y
y and a are not comparable
a
PARETO SET DEFINITION
feasible set attainable set
)1(x
)2(x
)1(f
)2(f
f
yYySWYyYP )(:)(
XY
)(YP*A
)()(*
YPAf
y
feasible set attainable set
)1(x
)2(x
)1(f
)2(f
F
me
Y
X y
y
-PARETO SET ILLUSTRATION
-PARETO SET DEFINITION
* *
is Pareto set if
1.
2. ( )
3.For each ( ) exists : m
Y
Y Y
P Y Y
y P Y y Y y e y
YAf
XA
solutionA
)(.2
,.1
:
meyy *
*( ) ( ) ( )mNE Y NE Y NE Y e
me
*( ( ), ( ))hd NE Y NE Y
*0
( ) ( )NE Y NE Y
24
Y
Y
Hausdorff
distance
hd
MCO WITH FUNCTIONAL CONSTRAINTS
)).(,),(()(,)(,),()(
:)(,:)(
where
},0)(:{
min,)(
)()1()()1( xgxgxGxfxfxF
RRGRRF
xGRxXx
xF
km
knmn
n
)(),( GF - continuous functions
-feasible and -attainable sets
nR mR)1(x
)2(x
)1(y
)2(y
FY
X
0)(: xGRxX n
0,)(: xGRxX n
)(XFY
)( XFY
YX
X
setfeasibleX
setfeasibleX
setattainableY
setattainableY
)1(y
)2(y
-Pareto Set ,
m1
Y
mm RYY
YYP
YY
Y
1.3
)(.2
1.
ifset Pareto,is
,
,,
,
,
YY
,,
,,
)(thatsuch
andset Pareto,find
0,0givenfor:goalThe
YXF
XY
COMPUTATIONAL EXPERIMENT
D. Chafekar, J. Xuan, K. Rasheed, Constrained Multiobjective
Optimization Using Steady State Genetic Algorithms, In Proceedings of
Genetic and Evolutionary Computation Conference, Chicago, Illinois, pp
813-824, 2003.
COMPUTATIONAL RESULTS
)1(y
)2(y
01.0,01.0
1040 iterations were
performed for 0.03 s
on Intel Xeon 2.6 GHz PC
COMBINING MAXIMIZATION AND
MINIMIZATION
max)(
min)(
)2(
)1(
xf
xf
)1(y
)2(y
Y
)(YP
)(* YPD
1
1
GENERALIZATION OF A NOTION OF
-PARETO SET
YDY
YYP
YY
*.3
)(.2
1.
Y
Y
)1(y
)2(y
Y
max)(
min)(
)2(
)1(
xf
xf
1
1
EFFICIENT FRONTIER
m
YPPeff
}1,1{
)(
)1(y
)2(y
Theorem. Efficient frontier of a
strictly convex set coincides
with its frontier.
1
1,
1
1,
1
1,
1
1
EFFICIENT HULL
m
YPDYH}1,1{
* )()(
)1(y
)2(y
Theorem. Convex hull of
a set contains its
efficient hull:
)()( YConvYHY
GENERALIZATION OF A NOTION OF
-PARETO SET
YDY
YYP
YY
*.3
)(.2
1.
Y
Y
)1(y
)2(y
Y
max)(
min)(
)2(
)1(
xf
xf
)1(y
)2(y
-EFFICIENT HULL
Y
m
YDH}1,1{
* )(
frontierfficientYB
n
e1,1
frontierefficientexternal
1,1
YBn
)1(y
)2(y
Y
)( BConvHY
B
B
-EFFICIENT HULL and CONVEX HULL
BOUNDING THE WORKING SET OF A
ROBOTIC MANIPULATOR
1
2
3
ROBOT WITH THREE LINKS
3
2
1
321
,36
,36
,25.0,2,3 LLL
SOLVING INEQUALITIES
nibax
xgxg
iii
m
,...,1],,[
0)(,...,0)(1
Find a set, defined by a system of inequalities
],[...],[ 11 nn babaP
NON-UNIFORM COVERINGS APPROACH
miallforxgiPx j
,...,10)(max
0)(min,...,1
xgmisomefor iPx j
Inner
Outer
Boundary otherwise subject to split
ALGORITHM OUTLINE
1:12
:11
:10
:9
:8
:7
)(:6
:5
0)(min,...,1:4
:3
,...,10)(max:2
:1
goto
endif
listthetothemaddand
boxestwointobisect
else
listthetoadd
thenifelse
listthetoadd
thensomeforifelse
listthetoadd
thenallforif
listthefromboxatakeelseexitthenif
L
P
BP
Pd
OP
xgmi
IP
mixg
LPL
j
j
j
j
iPx
j
iPx
j
j
j
Inner boxes
Boundary
boxes
Outer boxes
PRACTICAL EXAMPLE: APPROXIMATING
WORKING SET OF A ROBOT
𝑙0
1 2
𝑍
max
22
min
2
max
11
min
1
21
,
,
llllll
intervalsgiveninlengthchangecanlllinkages
whereZofpositionspossibleallFind
SYSTEM OF INEQUALITIES
𝑙0
1 2
𝑍
ANALYTIC SOLUTION
RESULTS
5.0
RESULTS
25.0
RESULTS
07.0
3-RPR Parallel Robot
SOFTWARE INFRASTRUCTURE
NUMERIC UTILITIES
INTERVAL ARITHMETICS
LIBRARY
EXPRESSIONS LIBRARY
NON-UNIFORM COVERING
METHOD “LOGIC”
SEQUENTIAL
SHARED MEMORY
DISTRIBUTED MEMORY
GPU (CUDA)
DISTRBUTED (BOINC)
EXPRESSIONS LIBRARY
Interval library supports only a limited set
of algebraic functions: +, -, sin, cos, ….
We need a support for complex
expressions, i.e.
n
k
xxkxxxf1
21121 )ln()sin(exp),(
EXPRESSIONS LIBRARY
• Allows to declare complex formulas
• Supports automated evaluation of a function value,
its derivatives, interval bounds and interval bounds
for derivates
ROSENBROCK FUNCTION
template <class T>
Expr<T> Rosenbrock(int n)
{
Expr<T> x;
Iterator i(0, n-2);
Expr<T> t = i;
return loopSum(100*sqr(x[t+1]-sqr(x[i]))+sqr((x[i]-1)), i);
}
DropWave function
template <class T>
Expr<T> DropWave()
{
Expr<T> x;
Expr<T> a = sqr(x[0])+sqr(x[1]);
Expr<T> b = 1+cos(12*sqrt(a));
Expr<T> c = 0.5*a + 2;
return –b/c;
}
Calling the calc method to get the value of the DropWave function at the given point (0.5, 1.5)
Expr<double> expr = DropWave<double>();
double result = expr.calc(FuncAlg<double>({0.5, 1.5}));
std::cout << result;
Calculation of the interval estimate of DropWave function over the box [0.5, 1.5]x[-
0.5, 0.5]
Expr<Interval<double>> expr = DropWave<Interval<double>>();
std::vector<Interval<double>> box =
{{ 0.5, 1.5}, {-0.5, 0.5}};
Interval<double> result = expr.calc(InterEvalAlg<double>(box));
std::cout << result;
Calculate the gradient of the DropWave function at the point (0.5,
1.5)
Expr<ValDer<double>> expr = DropWave<ValDer<double>>();
ValDer<double> result = expr.calc( ValDerAlg<double>({0.5, 1.5}));
std::cout << result;
Calculations of the interval estimation of the gradient of the DropWave function on a box
Expr<IntervalDer<double>> expr =
DropWave <IntervalDer<double>>();
std::vector<Interval<double>> box =
{{ 0.5, 1.5}, {-0.5, 0.5}};
IntervalDer<double> result = expr.calc(IntervalDerAlg<double>(box));
std::cout << result;
EXPRESSION LIBRARY
Expression
Internal Representation
Interval Bounds
Derivatives
LipschitzianBounds
Concave Minorants
SOFTWARE
The test suite for box-constrained
optimization
• https://github.com/alusov/mathexplib
Team
• Acad. D. Sc. Yury Evtushenko (scientific management)
• D.Sc. Mikhail Posypkin (general management)
• Ph.D. Andrey Gorchakov (parallel computing)
• Ph.D. Andrey Turkin (applications to robotics)
• Ph.D. student Alexander Usov (expression library)
• Ms. student Andrey Igantov (distributed computing)
Thank you for attention!
EXAMPLE PROBLEM
0 1
1
0 1
2
1
10
,10
)1()(
,)(
)2(
)1(
)2(2)1()2(
)1()1(
x
x
xxxf
xxf
Attainable set
Feasible set
(1)f
(2)f(2)x
(1)x
(1) (1) 2 (2)
1 2
1 2
( ) (( 1) )
0 ,0
v x x x x
Convolution function
IMPLEMENTATION
• Implemented using BNB-Solver library
– Y. Evtushenko, M. Posypkin, I. Sigal, A framework for parallel large-scale global optimization // Computer Science - Research and Development 23(3), pp. 211-215, 2009
• Publicly available https://github.com/mposypkin/BNB-solver
INFORMATION SOURCES
1. Initially proposed for global optimization problems in 1971Ю. Г. Евтушенко, “Численный метод поиска глобального экстремума функций
(перебор на неравномерной сетке)”, Ж. вычисл. матем. и матем. физ., 11:6 (1971),
1390–1403
2. Extened to multiobjective optimization in 1986
Евтушенко Ю. Г., Потапов М. А. «Методы численного решения
многокритериальных задач.» ДАН СССР, Т. 291, N 1, 1986, С. 25-39.
3. Parallel implementation 2009Y. Evtushenko, M. Posypkin, I. Sigal, A framework for parallel large-scale global
optimization // Computer Science - Research and Development 23(3), pp. 211-215, 2009..
4. Advanced techniques for mathematical programming problems 2013Y. Evtushenko, M. Posypkin. A deterministic approach to global box-constrained
optimization // Optimization Letters, 2013, Volume 7, Issue 4, pp 819-829.
5. Advanced techniques for multiobjective optimization Yu.G. Evtushenko. M.A. Posypkin, A deterministic algorithm for global multi-objective
optimization // Optimization Methods and Software, Vol. 29, Iss. 5, pp. 1005-1019, 2014
BRANCHING RULE (SIMPLIFIED)
B
B1 B2
Divide by the longest edge:
“ARCHIVING”
Procedure Update( , )
For all do
if ( ) ( ) then
\{ }
if ( ) ( ) then
exit
end do
{ }
A x
a A
F x F a
A A a
F a F x
A A x
a1
a2 a3
a4
Parameter space
BOUNDING CRITERIA FROM BELOW
• Introduce a minorant for each criterion:
• Compute lower bound on some box B’ - a point l(B’):
miBxxxf ii ,..,1,for)()( )()(
mixlllBl i
Bx
im ,...,1),(minwhere),...,()'( )(
'
)()()1(
BOUNDING RULE
'discard)1)(()'( BAFNEBl m
)1(x
)2(x
)1(y
)2(y
'B
)'(Bl
m1
archive
currentA
F
( ')F B
ALGORITHM OUTLINE
1. S = { X }, A = {}
2. If S is empty goto 8
3. Select (and remove) box B from S
4. Take point c – center of B
5. Update current archive A with c
6. Check bounding rule for B. If rule discards B then go to 2
7. Divide B into 2 boxes and add them to S, go to 2
8. End