classical optimization techniques -
TRANSCRIPT
-
7/25/2019 Classical Optimization Techniques -
1/26
CLASSICAL OPTIMIZATION
TECHNIQUES
-
7/25/2019 Classical Optimization Techniques -
2/26
Single Variable Optimization
A fn!tion of one "ariable f#$% i& &ai' to (a"e arelati"e or lo!al minimm at $)$* if f#$*% * f#$*+(
for all &ffi!ientl, &mall po&iti"e an' negati"e
"ale& of (- Similarl,. a point $*i& !alle' a
relati"e or lo!al ma$imm if $* if f#$*% / f#$*+(% foall "ale& of ( &ffi!ientl, !lo&e to zero-
A fn!tion f#$% i& &ai' to (a"e a global or
ab&olte minimm at $*if f#$
*
%* f#$% for all "aleof $ an' not 0&t !lo&e to $1.in t(e 'omaino"er
2(i!( f#$% i& 'efine'-
-
7/25/2019 Classical Optimization Techniques -
3/26
Single Variable Optimization
3elati"e an' 4lobal Minima
f#$%
$
a b
bo
A5
A6 A7
85
86
A5.A6. A7are relati"e ma$ima
A6i& global ma$imm
a b
3elati"e Minimm
I& 4lobal Minimm
f#$%
-
7/25/2019 Classical Optimization Techniques -
4/26
Single Variable Optimization
Similarl, a point $*2ill be a global ma$imm of f#$% if
f#$*
%/f#$% for all $ in t(e 'omain- A &ingle "ariable optimization problem i& one in 2(i!(
"ale of $)$*to be fon' ot in t(e inter"al 9a.b: &!( t$*minimize& f#$%- ;ollo2ing t2o t(eorem pro"i'ene!e&&ar, an' &ffi!ient !on'ition for t(e relati"eminimm of fn!tion of &ingle "ariable-
T(eorem I < Ne!e&&ar, Con'ition
If a fn!tion f#$% i& 'efine' in t(e inter"al a* $ * b an' a relati"e minimm at $) $1 . 2(ere a=$*=b. an' if t(e
'eri"ati"e
e$i&t& a& finite nmber at $)$1 t(en f> $* )?
)()( xfdx
xdf
-
7/25/2019 Classical Optimization Techniques -
5/26
Single Variable Optimization
Proof< It i& gi"en t(at
0if0)()f(x
0if0)()(
zero.tocloselysufficientvaluethefor)()haveweminimum,
relativeaisSincezero.betoproveto
wantwewhichnumber,definiteaasexists
)()(0)(
**
**
**
*
**lim
hh
xfh
hh
xfhxf
hxff(x
x
h
xfhxfhxf
-
7/25/2019 Classical Optimization Techniques -
6/26
Single Variable Optimization
maximum.relativeais
ifevenprovedbecantheorem1.his
!"otes
.theoremtheprovesthis
,0)(haveto
is(b)and(a)satisfytowayonlyhe
)(0)(
valuesne#ative
throu#h0hlimitthe#ivesitwhile
)(0)(
hofvaluespositivethrou#h0h$f
*
*
*
*
x
xf
bxf
axf
=
-
7/25/2019 Classical Optimization Techniques -
7/26
Single Variable Optimization
.applicablenotis
theoremtheexist,notdoes)(derivative
thee%ual,arenumberthe&nless
ly.respectivevaluesne#ativeorpositivethrou#h
zeroapproacheshwhetherondependin#
r)()(
0
slide.nexttheinshownfunctionthefor
exampleforexisttofailsderivativewhere
pointaatoccursmaximumorminimum
aifhappenswhatsaynotdoestheoremhe'.
!)(continued"otes
*
**
lim
*
xf
mandm
momh
xfhxfh
x
=
-
7/25/2019 Classical Optimization Techniques -
8/26
Negati"e &lope
Po&iti"e
&lope
f#$%
$
f#$1%
@1
-
7/25/2019 Classical Optimization Techniques -
9/26
Single Variable Optimization
7- T(e t(eorem 'oe& not &a, 2(at (appen& if a minimm
ma$imm o!!r& at an en' point of t(e inter"al of'efinition of t(e fn!tion- In t(i& !a&e
E$i&t& for all po&iti"e "ale& of ( onl, or all negati"e "aof ( onl,. an' (en!e t(e 'eri"ati"e i& not 'efine' at t(e
en' point-
-T(e t(eorem 'oe& not &a, t(at t(e fn!tion ne!e&&aril,
be ma$imm or minimm at e"er, point 2(ere t(e'eri"ati"e i& zero- ;or e$ample t(e 'eri"ati"e f>>#$%)? a
$)? for t(e fn!tion &(o2n in t(e ne$t &li'e- Ho2e"er
h
xfhxfh
)()(0
**lim
-
7/25/2019 Classical Optimization Techniques -
10/26
Single Variable Optimization
$
f#$%
Stationar, #infle!tion%
point f>#$%)?
O
;I4- STATIONA3B #IN;LECTION% POINT
-
7/25/2019 Classical Optimization Techniques -
11/26
Single Variable Optimization
If t(e fn!tion f#$% po&&e& !ontino&'eri"ati"e& of e"er, or'er t(at !ome in
e&tion. in t(e neig(bor(oo' of $)$51 t
follo2ing t(eorem pro"i'e& t(e &ffi!ient!on'ition for t(e minimm or ma$imm
"ale of t(e fn!tion-
-
7/25/2019 Classical Optimization Techniques -
12/26
10
)()()1(
)('
)()()(
haveweterms,n
afterremainderwiththeoremsaylor+pplyin#
odd.isnifminimumnormaximumaneither(c)even
nand0)(iff(x)ofvaluemaximuma(b)
evenisnand0)(iff(x)ofvalue
minimuma(a)is)(hen.0)(but
,0)()()(
letconditionSufficient!'heorem
**)1(1
*'
***
*
*
**
*)1(**