1. Problem Formulation

General Structure

• Objective Function: The objective function is usually formulated on the basis of economic criterion, e.g. profit, cost, energy and yield, etc., as a function of key variables of the system under study.

• Process Model: They are used to describe the interrelations of the key variables.

Example – Thickness of Insulation

Essential Features of Optimization Problems

1. At least one objective function, usually an economic model;

2. Equality constraints;3. Inequality constraints.

• Categories 2 and 3 are mathematical formulations of the process model.

• A feasible solution satisfies both the equality and inequality constraints, while an optimal solution is a feasible solution that optimize the objective function.

Mathematical Notation

1 2

max ( )

subject to

( )

( )

where

( )

T

n

f

x x x

h

xx

h x 0

g x 0

x

h x

1 1 2 2 1 2 1 2

1 1 2 2 1 2 1 2

, , , , , , , , ,

( ) , , , , , , , , ,

e

i

T

n n m n

T

n n m n

x x x h x x x h x x x

g x x x g x x x g x x x

g x

Objective function

Equality constraints

Inequality constraints

Economic Objective Function

Objective function

= income

- operating costs

- capital costs

EXAMPLE: OPTIMUM THICKNESS OF INSULATION

0

0 1

rate of heat loss

/ 1/

energy savings

( 0) ( )

/ 1/

cost of installed insulation

C

c

cc

A TQ

x k h

Q Q Q x Q x

A Th A T

x k h

C x

• The insulation has a lifetime of 5 years.• The fund to purchase and install the insulation

can be borrowed from a bank and paid back in 5 annual installments.

• Let r be the fraction of the installed cost to be paid each year to the bank. (r>0.2)

0 6

0 1

1/ 2

*6

1

kJ h dollars 1( ) year

h year 10 kJ

dollars

10

10

t

t

c

Hobj Q Q Y

r

C C x A

d obj H Y Tx k

dx kC r h

Time Value of Money

• The economic analysis of projects that incur income and expense over time should include the concept of the time value of money.

• This concept means that a unit of money on hand NOW is worth more than the same unit of money in the future.

Investment Time Line Diagram

Example

• You deposit $1000 now (the present value P) in a bank saving account that pays 5% annual interest compounded monthly.

• You plan to deposit $100 per month at the end of month for the next year

• What will the future value F of your investment be at the end of next year?

Present Value and Future Worth

If is the original investment (present value),

then (1 ) is the amount accumulated after

one compounding period, say 1 year. The value

of investment in n years for discrete interest

payments is

P

P i

(1 )

where is the future worth of investment after year n.

nn

n

F P i

F

Present Value of a Series of (not Necessarily Equal) Payments

11 22 1

n

k=1

1 1 1 1

1

n nn n

kk

F FF FP

i i i i

F

i

Present Value of a Series of Uniform Future Payments

1

Let each payment be and the first payment

starts in period m and the last payment ends in

period n

1 11

(1 )1

If 1,

1 1 1capital recovery factor

1

repayment multiplier

n

n mn

k nk m

n

F

iP F F

i ii

m

iP

F ri i

r

Repayment Multiplier

Future Value of a Series of (not Necessarily Equal) Payments

1

1

The future value at the end of ( 1) time period

1n

n k

kk

n

F F i

Future Value of a Series of Uniform Future Payments

1

Let each payment be and the first payment

starts in period m and the last payment ends in

period n, the future value at the end of period n+1 is

1 111

1

If 1,

1 1

n mnn

kk m

n

F

iF F i F

ii

m

iF

F i

Measures of Profitability(1) Payback period (PBP): how long a project must operate

to break even; ignores the time value of money:

cost of investment PBP =

cash flow per period

(2) Return of Investment (ROI): a yield calculation without

taking into account of the time value of money:

net income (after taxes) per year ROI (in percentage) = 100

cost of investment

Measures of Profitability

• Net present value (NPV) is calculated by adding the initial investment (represented as a negative cash flow) to the present value of the anticipated future positive (and negative) cash flows.

• Internal rate of return (IRR) is the rate of return (i.e. interest rate or discount rate) at which the future cash flows (positive plus negative) would equal the initial cash outlay (a negative cash flow).

2. Basic Concepts

Continuity of Functions

0

0

0

0

0

A function ( ) is continous at a point

if

(a) ( ) exists

(b) lim ( ) exists

(c) lim ( ) ( )

If ( ) is continuous at every point in a regi

x x

x x

f x x

f x

f x

f x f x

f x

on ,

then ( ) is said to be continous throughout .

R

f x R

Continuity of Functions

Stationary Point

0 0( ) 0 is a stationary point

A stationary point can be

- an extremum (i.e., maximum or minimum), or

- an inflection point

f x x

Unimodal and Multimodal Functions

• A unimodal function has one extremum.

• A multimodal function has more than one extrema.

• A global extremum is the biggest (or smallest) among a set of extrema.

• A local extremum is just one of the extrema.

2 2

1 2

1

2

1 2

1 2

min 3 4

subject to

0

0

5 0

2.5 0

f x x

x

x

x x

x x

2 2

1 2

1

2

1 2

1 2

min 2 2

subject to

0

0

5 0

2.5 0

f x x

x

x

x x

x x

Definition of Unimodal Function*

1 2 3 4

* *1 2 1 2

* *3 4 4 3

If is the point where ( ) reaches a maximum and,

for four arbitrarily selected points , , and ,

( ) ( ) ( )

( ) ( ) ( )

then ( ) is a uni

x f x

x x x x

x x x f x f x f x

x x x f x f x f x

f x

*

1 2 3 4

* *1 2 1 2

*3 4 4

modal function. Simlarly, if

is the point where ( ) reaches a minimum and,

for four arbitrarily selected points , , and ,

( ) ( ) ( )

( )

x

f x

x x x x

x x x f x f x f x

x x x f x f

*3( ) ( )

then ( ) is also a unimodal function.

x f x

f x

Convex and Concave FunctionsA function is called convex over a region R, if, for

any two values of x in R, the following inequality holds

2

2

2

2

1 1

where, is a scalar constant between 0 and 1.

Thus, if

0 for concave

0 for convex

a b a bf x x f x f x

d fa x b

dx

d fa x b

dx

Hessian Matrix

* * * *1 2 1 2

* * * *1 2 1 2

* *1 2

* * * *1 2 1 2 1 1 2 2

1 2, ,

2 * 2 2 * 21 1 2 2

2 21 2, ,

2

1 2 ,

Taylor series expalnsion of ( ) :

( , ) ( , ) ( ) ( )

( ) ( )

2! 2!

(

x x x x

x x x x

x x

f

f ff x x f x x x x x x

x x

f x x f x x

x x

f

x x

x

*

* *1 1 2 2

*

)( )

This equation can be written as

1( ) ( )

2

where is the Hessian matrix

TT

x x x x

ff f

x

x x x x H xx

H

Hessian Matrix

* * * *1 2 1 2

* * * *1 2 1 2

* * * *1 2 1 2

2

21 1 2, ,

2 2

21 2 2, ,

1 2, ,

*

At a stationary point,

0

Thus,

1( ) ( )

2

x x x x

x x x x

x x x x

T

f f

x x x

f f

x x x

f f

x x

f f

2

H

x x x H x

Positive and Negative Definiteness

A Hessian marix is positive definite (or positive semi-definite)

iff 0 (or 0) for all .

A Hessian marix is negative definite (or negative semi-definite)

iff 0 (or 0) for all .

T

T

x H x x 0

x H x x 0

A Hessian marix is indefinite if

0 for some and 0 for other T T x H x x 0 x H x x 0

Remarks

• A function is convex (strictly convex) iff its Hessian matrix is positive semi-definite (definite).

• A function is concave (strictly concave) iff its Hessian matrix is negative semi-definite (definite).

Tests for Strictly Convexity

1. All diagonal elements of Hessian matrix must be positive. Also, the determinants of Hessian matrix and all its leading principal minors must all be positive.

2. All eigenvalues of Hessian matrix must be positive.

Convex Region

A convex set of points exists, if for any two points

in the set (say and ), all points on the line

joining and , i.e. (1 ) and

0 1, are also in the same set.

a b

a b a b

x x

x x x x x

Convex Region

j i

j

j i

j

If a region is completely bounded by

g 0 j=1,2, ,m

and g s are concave functions, then the

bounded region is convex.

If a region is completely bounded by

g 0 j=1,2, ,m

and g s

x

x

x

x

are convex functions, then the

bounded region is convex.

Why do we need to discuss convexity and concavity?

• Determination of convexity or concavity can be used to establish whether a local optimal solution is also a global optimal solution.

• If the objective function is known to be convex or concave, computation of optimum can be accelerated by using appropriate algorithm.

Convex Programming Problem

min

s.t. 0 1, ,

in which

(a) is a convex function, and

(b) each inequality constraint is a convex function

(so the constraints form a convex set).

The local minimum of

i

f

g i m

f

f

xx

x

x

is also the global minimum!x

A NLP is generally not a convex programming problem!

The NLP problem

min

s.t.

0 1,2, ,

0 1,2, ,

may not be a convex programming problem if

any of the function is NONLINEAR.

Note that, if the function is nonlinea

i

k

k

k

f

g i m

h k r n

h

h

xx

x

x

x

x

r, then

0 represents a curved surface. Hence, the

line segment joining any two points on this surface

generally does not lie on the surface.

kh x

Proposition

Convex sets in satisfy the following relations:

(1) If is a convex set and is a real number, the

set : , is also convex;

(2) If and are convex sets, then the set

:

nR

C

C x x c c C

C D

C D x x c , ,

is also a convex set.

(3) The intersection of any collection of convex sets

is convex.

d c C d D

Linear Varieties

A set in is said to be a linear variety, if given

any , , we have (1 ) for all

real numbers .

A hyperplane in is an (n-1)-dimensional linear

variety.

n

n

V R

V V

R

1 2 1 2x x x x

Half Planes

Let be a non-zero n-dimensional column

vector, and be a real number. The set

: is a hyperplane in .

Let be a hyperplane in . Then there is

a non-zero n-dimensional vector and

a c

n T n

n

c

H R c R

H R

a

x a x

x

onstant such that

: .

The hyperplanes are convex sets.

n T

c

H R c x a x

Half Spaces

Let be q nonzero vector in and let

be a real number. Corresponding to the

hyperplane : , the positive

and negative closed half spaces are:

: and :

and the positive and neg

n

T

T T

R c

H c

H c H c

a

x a x

x a x x a x

ative open half spaces

are:

: and :T TH c H c x a x x a x

Clearly the half spaces are convex sets.

Polytope and Polyhedron

• A set which can be expressed as the intersection of a finite number of closed half spaces is said to be a convex polytope.

• A nonempty bounded polytope is called a polyhedron.

Necessary Conditions for the Extremum of an Unconstrained Fun

ction

*f x 0

Implies one of the following three possibilities:(1) a minimum, (2) a maximum or (3) a saddlepoint.

* *

* * *

* *

* * 2 *

* * *1 1 2 2

*

1

2 2 2

1 1 1 2 1

2 *

2 2

1

1( ) ( ) ( )

2where

( )

T T

T

n n

T

n

n

n n n

f f f f

x x x x x x

f ff

x x

f f f

x x x x x x

f

f f

x x x x

x x

x x x

x x

x x x x x x x

x

x

x

*

*

*

* *

If is a local minimum, then

( ) ( ) 0 (A)

in the neighborhood of . First let us consider

the 1st term in the Taylor series

( ) ( ) ( )

Since equation (

T

f f

f f f

x

x x

x

x x x x

*

*

*

A)should be satisfied for all points

in the vicinity of , should be arbitrary in

direction. If ( ) 0, then one can always

find a such that ( ) 0.

This violates equation (A)! There

T

T

f

f

x x

x

x x x

*

fore, one requires

( ) 0

.

T f x

Sufficient Conditions for the Extremum of an Unconstrained Funct

ion

* 2 *

*

* 2 *

*

or is positive definite

local minimum!

or is negative definite

local maximum!

f

f

H x x

x

H x x

x

Theorem

Suppose at a point the first derivative is zero and the first nonzero higher-order derivative is denoted by n,

1. If n is odd, this point is a inflection point.

2. If n is even, then it is a local extremum. Moreover,

*

*

*

*

(a) If >0, then is a local minimum

(b) If <0, then is a local maximum

n

n

x

n

n

x

d fx

dx

d fx

dx