Optimization

from Prof. Goldsman’s lecture notes

Outline

What is a Model?What is a Model?

Operation ResearchOperation Research

Optimization ExampleOptimization Example

Shortest PathShortest Path

Introduction to ModelingIntroduction to Modeling

What is a Model?

Abstraction, representationAbstraction, representation

Infinitely many models of the same realityInfinitely many models of the same reality

Often a model is created for a purposeOften a model is created for a purposeA good model discards the irrelevantA good model discards the irrelevantA good model retains what is crucialA good model retains what is crucial

Often we believe we understand something Often we believe we understand something better after modeling itbetter after modeling it

We trust a model if it gives accurate predictions We trust a model if it gives accurate predictions (qualitative or quantitative)(qualitative or quantitative)

Note that…When you want to create a When you want to create a model…model…

Collect the necessary dataCollect the necessary data

Distinguish between the Distinguish between the model model inputinput and and model outputmodel output

Lengthy notes are worthlessLengthy notes are worthless

KISS RuleKISS Rule

Operation ResearchHigher level of theoretical and Higher level of theoretical and mathematical orientationmathematical orientation

Categorization of Operation Research:Categorization of Operation Research:

Wants to model the problemWants to model the problemEncounter the problems of estimating the Encounter the problems of estimating the values of the parametersvalues of the parametersParameters may not be constant over timeParameters may not be constant over time

2 Approaches:2 Approaches:Forget the fact that they are RV and use the Forget the fact that they are RV and use the model w/o RV taken into consideration – model w/o RV taken into consideration – Deterministic ApproachDeterministic ApproachUse model w/ RV – Use model w/ RV – Probabilistic ApproachProbabilistic Approach

Optimization ExampleShortest Path of Auto Travel Shortest Path of Auto Travel RoutesRoutes

Distances are in milesDistances are in miles

SS

aa

bb

cc

dd

tt

100

150

20

100

8070

200

180

40

Shortest Path (Cont’d)Optimal Solution has length 270 Optimal Solution has length 270 milesmiles

But it did not go to node But it did not go to node d d and and aa

SS

aa

bb

cc

dd

tt

100

150

20

100

8070

200

180

40

Shortest Path (Cont’d)

Algorithm actually finds aAlgorithm actually finds a tree tree giving giving shortest paths from shortest paths from ss to to everyevery nodenode in graph in graph

SS

aa

bb

cc

dd

tt

100

150

20

100

8070

200

180

40

Shortest Path: Definitions

Graph G= (V,E)Graph G= (V,E)V: vertex set, contains special vertices s and tV: vertex set, contains special vertices s and tE: edge setE: edge set

Costs Cij on edges (i, j) in ECosts Cij on edges (i, j) in ECij >= 0Cij >= 0no cycles with negative total costno cycles with negative total cost

Cost of a path = sum of edge costsCost of a path = sum of edge costsObjectiveObjective: find : find min cost pathmin cost path from from ss to to tt

Shortest Path (Cont’d)A Mathematical ProblemA Mathematical ProblemAn Optimization ProblemAn Optimization Problem

It has: It has: A set of possible solutionsA set of possible solutions (paths from (paths from ss to to tt))An objective functionAn objective function (min the sum of edge costs) (min the sum of edge costs)

An An algorithmalgorithm that that correctlycorrectly and and quicklyquickly solves solves cases of the shortest path problem, provided thatcases of the shortest path problem, provided that

The instances satisfy Cij >= 0The instances satisfy Cij >= 0The instances are not too hugeThe instances are not too huge

ExampleGoal:Goal: use a car for 4 years at min cost use a car for 4 years at min cost

Purchase Cost

1st year maint.

1 year Resale

2nd year maint

2 year Resale

3rd year maint

3rd year Resale

New Car

15000 1000 11000 1000 9000 1500 8000

Used Car

5000 2000 4000 3000 3000 3500 2500

Example (Cont’d)

Vertices of graph need not represent Vertices of graph need not represent physical locationsphysical locations

V= {0,1,2,3,4}V= {0,1,2,3,4}time 0, 1,...,4 in yearstime 0, 1,...,4 in years

Seek least expensive path from 0 to 4Seek least expensive path from 0 to 4

Edge cost from i to j: cost of buying a car Edge cost from i to j: cost of buying a car at time i, using it, and selling it at time jat time i, using it, and selling it at time j

for each edge, pick cheapest alternative (new for each edge, pick cheapest alternative (new or used)or used)

Example (Cont’d)

Keep the new car for 1 year: Keep the new car for 1 year: 50005000Keep the used car for 1 year: Keep the used car for 1 year: 30003000Keep the new car for 2 years: Keep the new car for 2 years: 80008000Keep the used car for 2 years:Keep the used car for 2 years: 7000 7000

0 41 32

Buy at time 0, keep2 years, sell at time 2

Buy at 1,sell at 2

Buy at 0sell at 1

Example (Cont’d)

Continue and find the shortest pathContinue and find the shortest path

Note thatNote that: shortest path does allow : shortest path does allow directed graphdirected graph

i.e. you cannot go from node 2 to i.e. you cannot go from node 2 to node 1 at a cost of 3000 node 1 at a cost of 3000

Optimization Model Usage

Real problem

Real problem

Math Problem(Optimization Model)

Math Problem(Optimization Model)

Solution to Math Problem

Solution to Math Problem

DataData

AlgorithmConceptual

ModelConceptual

Model

What is A Successful Model?

Models must fit the real problemModels must fit the real problem

Realism and GeneralityRealism and Generality

Able to solve the model Able to solve the model

Solvability and TractabilitySolvability and Tractability

Modeling FAQ

Tradeoff betweenTradeoff between realismrealism and and solvabilitysolvability

Good modelers know … Good modelers know … Different models limitationsDifferent models limitationsFitting a model into a wider range of real Fitting a model into a wider range of real problemsproblemsFitting a real problem into a modelFitting a real problem into a model

Advanced modelers know how to …Advanced modelers know how to …Solve a wider range of modelsSolve a wider range of modelsExtend the range of cases that can be solved Extend the range of cases that can be solved with software toolswith software tools

Optimization Models Spectrum

Networks LP Convex QP IP NLPNetworks LP Convex QP IP NLPShortest Path

Min Span TreeMax FlowAssignmentTransportationMin Cost Flow

Shortest PathMin Span TreeMax FlowAssignmentTransportationMin Cost Flow

portfoliooptimizationportfoliooptimization

chemicalprocesses

materialsdesign

chemicalprocesses

materialsdesign

blending

planning

blending

planning logistics

scheduling

logistics

scheduling

production/distributionflow of materialsproduction/distributionflow of materials

Mathematical ProgrammingLinear ProgrammingLinear Programming

Nonlinear ProgrammingNonlinear ProgrammingInteger ProgrammingInteger ProgrammingBinary ProgrammingBinary ProgrammingQuadratic ProgrammingQuadratic ProgrammingGeometric ProgrammingGeometric ProgrammingDynamic ProgrammingDynamic ProgrammingMixed Integer ProgrammingMixed Integer Programming

Mathematical Programming (Cont’d)Three important characteristics:Three important characteristics:

Decision VariablesDecision VariablesObjective FunctionObjective FunctionConstraintsConstraints

Solving:Solving:GraphicallyGraphicallySimplex Simplex

A Piece of Cake ExampleFarmer Jones must determine how many Farmer Jones must determine how many

acres of corn and wheat to plant this year. An acres of corn and wheat to plant this year. An acre of wheat yields 25 bushels of wheat and acre of wheat yields 25 bushels of wheat and requires 10 hours of labor per week. an acre requires 10 hours of labor per week. an acre of corn yields 10 bushels of corn and requires of corn yields 10 bushels of corn and requires 4 hours of labor per week. All wheat can be 4 hours of labor per week. All wheat can be sold at $4 a bushel and all corn can be sold at sold at $4 a bushel and all corn can be sold at $3 a bushel. Seven acres of land and 40 hours $3 a bushel. Seven acres of land and 40 hours per week of labor are available. Government per week of labor are available. Government regulations require that at least 30 bushels of regulations require that at least 30 bushels of corn can be produced during the current corn can be produced during the current year. year.

Formulate an LP whose solution will tell Formulate an LP whose solution will tell Farmer Jones how to maximize the total Farmer Jones how to maximize the total revenue from wheat and corn.revenue from wheat and corn.

Find the solution using graphical method.Find the solution using graphical method.

Assignment ProblemAssignment ProblemAssignment Problem

Simplest and easiestSimplest and easiest

Def: there are N items or services Def: there are N items or services available at N locations and N other available at N locations and N other locations that require one and only one of locations that require one and only one of these N items or servicesthese N items or services

Example: 4 types of product and 4 Example: 4 types of product and 4 machines. Each machine can only machines. Each machine can only produce one type of products.produce one type of products.

Objective: assign jobs to machines that Objective: assign jobs to machines that will will minimizeminimize the total cost the total cost

Assignment Problem (Cont’d)

For unbalanced assignment For unbalanced assignment problemsproblems

Add dummy rows or columnsAdd dummy rows or columns

Set the cost of these rows or Set the cost of these rows or columns to be all zeros columns to be all zeros

Solving Assignment ProblemUsing Using Hungarian MethodHungarian Method

Subtract the smallest number in each row from Subtract the smallest number in each row from every entry in that rowevery entry in that rowSubtract the smallest number in each column Subtract the smallest number in each column from every entry in that columnfrom every entry in that columnA zero cost assignment (if it can be made) is A zero cost assignment (if it can be made) is optimal. If not, complete the Hungarian Methodoptimal. If not, complete the Hungarian MethodDraw the min # of horizontal and vertical lines Draw the min # of horizontal and vertical lines (no diagonals) that intersect all the zeros(no diagonals) that intersect all the zerosSubtract the smallest uncrossed number from Subtract the smallest uncrossed number from every other uncrossed number and add it to all every other uncrossed number and add it to all elements where the vertical and horizontal lines elements where the vertical and horizontal lines intersectintersectIf a zero cost assignment can be made, it is If a zero cost assignment can be made, it is optimaloptimalO’w repeat steps 4 and 5 until one can be madeO’w repeat steps 4 and 5 until one can be made

Solving Assignment Problem (Cont’d)Draw the min # of Draw the min # of

horizontal and vertical horizontal and vertical lines (no diagonals) that lines (no diagonals) that intersect all the zerosintersect all the zeros

Subtract the smallest Subtract the smallest uncrossed number from uncrossed number from every other uncrossed every other uncrossed number and add it to all number and add it to all elements where the elements where the vertical and horizontal vertical and horizontal lines intersectlines intersect

1 2 3 4 5

1 7 3 2 0 0

2 4 0 1 1 1

3 0 0 0 1 1

4 1 1 1 3 0

5 1 1 1 6 0

1 2 3 4 5

1 7 3 2 0 1

2 4 0 1 1 2

3 0 0 0 1 2

4 0 0 0 2 0

5 0 0 0 5 0

Smallest no = 1Row 4 & 5: Subtract 1Intersection: add 1

Solving Assignment Problem (Cont’d)

1 2 3 4 5

1 7 3 2 0 1

2 4 0 1 1 2

3 0 0 0 1 2

4 0 0 0 2 0

5 0 0 0 5 0

If a zero cost assignment can be made optimal

Solution?4-1, 2-2, 3-3, 1-4, 5-5

Hungarian Method: MaxObjective: maximize Objective: maximize

Note that max f(x) = min –f(x)Note that max f(x) = min –f(x)

Solve:Solve:Replace all the cost elements by Replace all the cost elements by their negativetheir negative

Continue w/ the minimization Continue w/ the minimization procedureprocedure

Hungarian Method: Max (Cont’d)Replace all the cost elements by their negativeReplace all the cost elements by their negative

Subtract the smallest number in each row from Subtract the smallest number in each row from every entry in that row (perform similar calculation every entry in that row (perform similar calculation for column)for column)

1 2 3 4

1 5 3 5 4

2 2 1 6 3

3 5 4 6 2

4 6 3 1 4

1 2 3 4

1 -5 -3 -5 -4

2 -2 -1 -6 -3

3 -5 -4 -6 -2

4 -6 -3 -1 -4

1 2 3 4

1 0 2 0 1

2 4 5 0 3

3 1 2 0 4

4 0 3 5 2

Find the Find the smallest smallest number number per rowper row

1 2 3 4

1 0 0 0 0

2 4 3 0 2

3 1 0 0 3

4 0 1 5 1

Find the Find the smallest smallest number number per per columncolumn

Transportation ProblemThere are M sources with There are M sources with

something available and N something available and N destinations needing somethingdestinations needing something

Difference:Difference:Assignment Problem: Assignment Problem: f(x): x f(x): x y yTransportation problem: Transportation problem:

f(x): x f(x): x y y11, y, y22,…,y,…,ynn

f(x): xf(x): x11, x, x22,…,x,…,xmm y y

Transportation Problem (Cont’d)Mathematical Formulation:Mathematical Formulation:

aaii = number of units available at i = number of units available at i

bbjj = #units needed at j = #units needed at j

ccijij = cost to ship 1 unit from i to j = cost to ship 1 unit from i to j

xxijij = #units shipped from source i to destination j = #units shipped from source i to destination j

M N

ij iji=1 j=1

M

ij ji=1

N

ij ij=1

N M

j ij=1 i=1

Optimize z = c x

subject to x = b j = 1,…, N

x = a i = 1,…,M

where b = a

More ExampleSmallco, Inc.Smallco, Inc.Smallco, Inc. manufactures two products, Smallco, Inc. manufactures two products, wooden toy cars and wooden toy trucks, wooden toy cars and wooden toy trucks, made from mahogany. The profit margin made from mahogany. The profit margin for the two toys is $1.10 and $0.70, for the two toys is $1.10 and $0.70, respectively. Based on careful market respectively. Based on careful market analysis, it appears that Smallco can sell analysis, it appears that Smallco can sell about 2000 of each toy each week. about 2000 of each toy each week. However, Smallco can only obtain a limited However, Smallco can only obtain a limited amount of mahogany, roughly 3000 board-amount of mahogany, roughly 3000 board-feet per week. Producing either toy feet per week. Producing either toy requires 1 board foot of wood. requires 1 board foot of wood. What is the best mix of toys for Smallco to What is the best mix of toys for Smallco to produce if the goal is to maximize total produce if the goal is to maximize total profit margin?profit margin?

More Example (Cont’d)What is the objective?What is the objective?

What can be controlled?What can be controlled?

How do the decisions affect the How do the decisions affect the objective?objective?

What limits the decisions?What limits the decisions?

The LP model is …The LP model is …

Solve it!Solve it!

Example… AgainDorial Auto manufactures luxury cars and Dorial Auto manufactures luxury cars and trucks. The company believes that its most trucks. The company believes that its most likely customers are high-income women and likely customers are high-income women and men. To reach these groups, Dorial Auto has men. To reach these groups, Dorial Auto has embarked on an ambitious TV advertising embarked on an ambitious TV advertising campaign, and has decided to purchase 1-campaign, and has decided to purchase 1-minute ad spots on two types of programs; minute ad spots on two types of programs; comedy shows and football games. Each comedy shows and football games. Each comedy commercial is seen by 7 million high-comedy commercial is seen by 7 million high-income women and 2 million high-income income women and 2 million high-income man. man. A 1-minute comedy spot costs $50,000 and a A 1-minute comedy spot costs $50,000 and a 1-minute football spot costs $100,000. Dorial 1-minute football spot costs $100,000. Dorial wants to reach at least 28 million high-income wants to reach at least 28 million high-income women and 24 million high-income man. How women and 24 million high-income man. How should Dorian buy commercial time to reach should Dorian buy commercial time to reach their target at the lowest possible cost?their target at the lowest possible cost?

Model With Discrete Variables

Model with specific (discrete) Model with specific (discrete) values, values, is not a linear is not a linear programming modelprogramming model

Types:Types:Integer ProgrammingInteger Programming

Mixed Integer ProgrammingMixed Integer Programming

Binary Integer ProgrammingBinary Integer Programming

Model With Discrete Variables (Cont’d)When do you need discrete variables?When do you need discrete variables?

Continuous problemsContinuous problems““yes/no” aspects to the decisionyes/no” aspects to the decision““How many” not “How much”How many” not “How much”

How to model it?How to model it?Non-obvious “trick”Non-obvious “trick”Treat each linear function as a separate cost Treat each linear function as a separate cost function, with its own “variablefunction, with its own “variable””Associate a binary variable with each of the Associate a binary variable with each of the linear functionlinear functionEnforce the requirement that you cannot do Enforce the requirement that you cannot do this if you don’t have this (TRICK)this if you don’t have this (TRICK)

Model With Discrete Variables (Cont’d)It is trickyIt is tricky

It is not obvious and most of us It is not obvious and most of us (including myself) would have a (including myself) would have a hard time “inventing” this ourselveshard time “inventing” this ourselvesCan be very difficult to solveCan be very difficult to solveExample: Example: selecting among selecting among investment alternatives, designing investment alternatives, designing supply chains, sourcing products, supply chains, sourcing products, production/inventory planning, production/inventory planning, crew scheduling, vehicle routing, crew scheduling, vehicle routing, etcetc