lecture 1

E i 172Economics 172Introduction to Operation Research

(Part 2)

1. Introduction

Spring 2009Spring 2009Herb Newhouse

1

ReadingsReadings

• Hillier & Lieberman (8th edition)Hillier & Lieberman (8 edition)– Ch. 12: Intro.

12 1: Sample Applications– 12.1: Sample Applications.– 12.2: Graphical Illustration on Nonlinear

Programming ProblemsProgramming Problems.– 12.3: Types of Nonlinear Programming

ProblemsProblems.– Appendix 2: Convexity

2

OutlineOutline

• Introduction to non-linear programming.Introduction to non linear programming.– Unconstrained optimization.– Equality constrained optimization.Equality constrained optimization.– Inequality constrained optimization.

• Concavity & ConvexityConcavity & Convexity– Concave/convex functions of one variable.– Concave/convex functions of multiple p

variables.– Convex sets.

3

Non-Linear ProgrammingNon Linear Programming

• In economics applications we often runIn economics applications we often run into non-linear optimization problems.

• The basic objective is the same as in linear programming.linear programming.

• The techniques are different than those• The techniques are different than those used in linear programming because we may get interior solutions.may get interior solutions.

4

Unconstrained OptimizationUnconstrained Optimization

• We want to find the global (or local)We want to find the global (or local) maximum or minimum for a function.

• For instance, a monopolist may wish to h tit t d t i ichoose a quantity to produce to maximize

profit.

5


max f x

maxx

f x

x x x x

1 2, , nx x x x

6


f f x

x

7

Equality ConstrainedEquality Constrained

• We want to find the maximum or minimumWe want to find the maximum or minimum for a function subject to an equality constraintconstraint.

F i t i i tilit bj t t• For instance, maximize utility subject to a budget constraint.– Or minimize the cost of producing a given

amount of output.

8


max subject to for 1 2i if x g x b i m

max subject to for 1, 2, ,i ixf x g x b i m

x x x x

1 2, , nx x x x

9


2x

100

1x

1 2, 100g x x

10


2x

b

1x

1 2,g x x b

11

Inequality ConstrainedInequality Constrained

• We want to find the maximum or minimumWe want to find the maximum or minimum for a function subject to an inequality constraint.

• For instance, maximize utility subject to aFor instance, maximize utility subject to a budget constraint (that doesn’t have to be completely spent).p y p )– Or minimize the cost of producing at least a

given amount of output.

12


max subject to for 1 2i if x g x b i m

max subject to for 1, 2, ,i ixf x g x b i m

x x x x

1 2, , nx x x x

13


2x

1x

1 1 2, 100g x x 2 1 2, 150g x x

14


2x

1 1 2 1,g x x b

g x x b

1x

2 1 2 2,g x x b

15


2x

1 1 2 1,g x x b

1x 2 1 2 2,g x x b

16

Concavity and ConvexityConcavity and Convexity

17

Concavity and ConvexityConcavity and Convexity

• We use the terms concave and convex toWe use the terms concave and convex to describe the curvature of a function.

• We say f(x) is concave if the entire line• We say f(x) is concave if the entire line segment connecting any two points on the function lies below the functionfunction lies below the function.– We can test this with the second derivative.

A f ti i if f ll i''( ) 0f– A function is concave if for all x in the domain of f(x)

''( ) 0f x

18

Convex FunctionsConvex Functions

• We say f(x) is convex if the entire lineWe say f(x) is convex if the entire line segment connecting any two points on the function lies above the functionfunction lies above the function.– We can test this with the second derivative.

A function is convex if for all x in the''( ) 0f x – A function is convex if for all x in the domain of f(x)

( ) 0f x

19

A Concave FunctionA Concave Function

2 4y x x

20

A Concave FunctionA Concave Function• We can also show this function is negative by g y

determining the sign of the second derivative.

22 4' 2 4

y x xy x

'' 2 0y

21

Concavity and ConvexityConcavity and ConvexityFormally:

A function is concave ifg x

1 1

for all and for any 0 1.

g x y g x g y

x y

A function is convex ifg x 1 1


g x y g x g y

x y

yy

22

A Concave Function

2 4g x x x

23

A function is concave if

1 1

g x

b b 1 1

2

g a b g a g b

2 24 , 1, 4,3

g x x x a b

2 21 1 1 4 23 3

a b

21 2 2 4 2 4g a b g

2 21 1 4 1 3, 4 4 4 4 0

2 2

g a g g b g

2 21 3 1 0 23 3

g a g b 24

Strict/Weak ConcavityStrict/Weak ConcavityIf the inequalities are strict we say the function is strictly concave.

If the inequalities are weak we say the function is weakly concave.

A f i i i l if '' 0 f ll i h df i A function is strictly concave if '' 0 for all in the domaf x x

in.

A function is weakly concave if '' 0 for all in the domain.f x x

A function is strictly concave ifg x

1 1


g x y g x g y

x y

A function is weakly concave if we have in t he above expression.25

Strict/Weak ConvexityStrict/Weak ConvexityIf the inequalities are strict we say the function is strictly convex.

If the inequalities are weak we say the function is weakly convex.

A f i i i l if '' 0 f ll i h d if A function is strictly convex if '' 0 for all in the domain.f x x

A function is weakly convex if '' 0 for all in the domain.f x x

A function is strictly convex ifg x

1 1


g x y g x g y

x y

A function is weakly convex if we have in the abo ve expression.26

Example 1Example 1

f x 2 8 16f x x x

f x

x

27

Example 2Example 2

f x 4f x x

f x

x

28

Example 3Example 3

f x 4f x x

f x

x

29

Example 4

3 21 3

Example 4

3 21 3( ) 23 2

f x x x x

30

Concave/Convex RegionsConcave/Convex RegionsEven if a function is not entirely concave or convex

it still may be concave or convex over a certainregion.

Example 4:3

3'' 2 3 023'' 2 3 0

f x x x

f

3'' 2 3 02

3 3is concave when and convex when

f x x x

f x x

is concave when and convex when .2 2

f x x

31

Multivariate FunctionsMultivariate Functions

• We have analogous definitions when theWe have analogous definitions when the functions are of more than one variable.

The function is concave if the line connecting– The function is concave if the line connecting any two points on the function is below the function.

• Strictly concave if the line is strictly below the function.

• Weakly concave if the line is weakly below the function.

For convex replace ‘below’ with ‘above’– For convex, replace below with above .32

Multivariate FunctionsMultivariate FunctionsFormally:

A function is concave ifg x

1 1


g x y g x g y

x y

y

A function is convex if

y

g x 1 1

f ll d f 0

g

g x y g x g y

1for all and for any 0x y 1.

33

Second Order ConditionsSecond Order Conditions

• We can generalize our test for functions ofWe can generalize our test for functions of more than one variable using the Hessianmatrixmatrix.– Matrix of all second order partials.

34

HessianHessian

1 2, , , nf f x x xx

'' ij n nf f

H x x

11 12 1

21 22 2

n

n

f f ff f f

x x xx x x

21 22 2

1 2

n

n n nn

f f f

f f f

H

x x x

1 2n n nn

35

Example 2 2

1 2 1 2 1 2, 4f x x x x kx x

1 1 2 2 2 21 2

2 ; 8f ff x kx f x kxx x

2 2

11 212 2;f ff f k 2

1 2 12 2

12 22 2; 8

x x xf ff k f

x x x

1 2 2

2

x x x

f f k

11 12

21 22

28

f f kf f k

H36

Unconstrained ProblemsUnconstrained Problems

• We’re going to use the Hessian’s leadingWe re going to use the Hessian s leading principal minors to express the second order conditionsorder conditions.– Determinants starting from the top left

element of H and gradually expanding toelement of H and gradually expanding to cover the entire matrix.

37

Leading Principal Minors of HLeading Principal Minors of H

11 12 1

21 22 2

n

n

f f ff f f

x x xx x x

H

1 2n n nnf f f

H

x x x

38


12 1

21 22 2

11 n

n

f ff

ff f

x xxx x x

H

21n n nnf ff

H

x x x

39


11 12

21 2

1

22

n

n

ff

f ff f

xx

H

x xx x

1 2n n nnf f f

H

x x x

40


11 12 1

21 22 2

n

n

f f ff f f

x x xx x x

H

1 2n n nnf f fx x x

H

41


11 12 1rf f fx x x

21 22 2r

r

f f fD

x x xx

1 1r r rrf f fx x x

1,2, ,r n

42

ConcaveConcave

1 0rrD x

1The sign of is negative.D xThe signs alternate as we increase .r

is negative definite.H

43

ConcaveConcave

12 111 nf ff

x x x 2 21 22 nf f f

x x xH

1 2n n nnf f f x x x

1 0D x

44

ConcaveConcave

11 12 1nff f

xx x 22 221 nff f

xH

x x 1 2n n nnf ff

x x x

2 0D x

45

ConcaveConcave

11 12 1nf f fx x x

21 22 2nf f f

x x xH

1 2

if i dd

n n nnf f fx xx

0 if is odd.0 if is even.

n

n

D nD n

xx

46

ConvexConvex

0rD x

The sign of each is positive.rD x g p

is positive definite

r

H is positive definite.H

47

Example (continued)

2 21 2 1 2 1 2, 4f x x x x kx x

28

kk

H8

2 t b

k

D f

1

2 22

2

2 cannot be convex.

2 8 16 0

D fD k k

2 16

4 4 is concave.If 4 4 th i ith

kk f

k k f

If 4 or 4 then is neither concave nor convex.k k f 48

Convex SetsConvex Sets

Let and be any two points in a set.The set is convex if the entire line connecting and

x yx yThe set is convex if the entire line connecting and

is also in the set.x y

If and , then for all 0 1, 1 .x S y S x y S

49

Examples 1 and 2Examples 1 and 2

1

1

All real numbers | 0 4 is a convex set.

S x xS

2 All integers | 0 4S x x

2 is in the set.3 is in the set.2 25 i t

2

2.25 is not. is not a convex set.S

50

Examples 3 and 4Examples 3 and 4

3S

4S

51

lecture 1

Documents