IE 312 Review

1

The Process

2

Problem

Model

Conclusions

Problem Formulation

Analysis

Problem Formulation

3

What is the objective? Maximize profit, Minimize inventory, ...

What are the decision variables? Capacity, routing, production and stock levels

What are the constraints? Capacity is limited by capital Production is limited by capacity

Analysis

4

Optimization Algorithm Computer Implementation

Excel (or other spreadsheet)

Optimization software (e.g., LINDO)

Modeling software (e.g., LINGO)

IncreasingComplexity

Optimization Algorithms

5

Find an initial solution Loop:

Look at “neighbors” of current solution Select one of those neighbors Decide if to move to selected solution Check stopping criterion

Tractability and Validity

6

Is a model tractable? Can it be solved, or is it too complex and/or large?

Is a model valid? Do we reach the same conclusions experimenting

with the model as we would experimenting with the real system?

Mathematical Programming

7

General Model Minimize or maximize some objective function Subject to some constraints

Linear versus Nonlinear

8

A function is linear if it is a weighted sum of the decision variables, otherwise nonlinear

A linear program (LP) has a linear objective function f and constraint functions g1,…,gm

A nonlinear program has at least one nonlinear function

Integer Programs

9

A variable is discrete if it can only take a limited or countable number of values

A variable is continuous if it can take values in a specific interval

Mathematical programs can be continuous discrete (integer/combinatorial) mixed

Classification Summary

10

Linear Program (LP) ILP

Nonlinear Program (NLP) INLP

Integer Program (IP)

Increased difficulty

Solution Techniques

11

Models Thousands (or millions) of variables Thousands (or millions) of constraints

Complex models tend to be valid Is the model tractable?

Solution techniques

Improving Search

12

Begin at feasible solution Advance along a search path Ever-improving objective function value

Neighborhood: points within small positive distance of current solution

Stopping criterion: no improvement

Local Optima

13

Improving search finds a local optimum

May not be a global optimum(only a heuristic solution)

Tractability: for some models there is only one local optimum (which hence is global)

Tractability

14

The most tractable models for improving search are models with unimodal objective function (linear special case) convex feasible region (linear special case)

Here every local optimum is global

Solving LP Models

15

Improving Search Unimodal Convex feasible region Should be successful!

Special Form of Improving Search Simplex method

Optimal Solutions

16

Every optimal solution is a boundary point We can find an improving direction whenever

we are at an interior point If optimum unique the it must be an

extreme point of the feasible region If optimal solution exist, an optimal

extreme point exists

LP Standard Form

17

Easier if we agree on exactly what a LP should look like

Standard form only equality main constraints only nonnegative variables variables appear at most once in left-hand-side

and objective function all constants appear on right hand side

Extreme Points

18

Know that an extreme point optimum exists Will search trough extreme points

An extreme point is define by a set of constraints that are active simultaneously

Improving Search

19

Move from one extreme point to a neighboring extreme point This defines the directions Simplex is a special case of improving search

that only uses these directions Extreme points are adjacent if they are

defined by sets of active constraints that differ by only one element

An edge is a line segment determined by a set of active constraints

Basic Solutions

20

An easy way of keeping track of the Simplex directions is by using some linear algebra

Extreme points are defined by set of active nonnegativity constraints

A basic solution is a solution that is obtained by fixing enough variable to be equal to zero, so that the equality constraints have a unique solution

Simplex Algorithm

21

Starting point A basic feasible solution (extreme point)

Direction Follow an edge to adjacent extreme point:

Increase one nonbasic variable Compute changes needed to preserve equality

constraints One direction for each nonbasic variable

Sensitivity Analysis

22

Basic Question: How does our solution change as the input parameters change? The objective function?

More/less profit or cost The optimal values of decision variables?

Make different decisions! Why?

Only have estimates of input parameters May want to change input parameters

What We Know

23

Qualitative Answers for All Problems Quantitative Answers for Linear Programs

(LP) Dual program Same input parameters Decision variables give sensitivities Dual prices Easy to set up Theory is somewhat complicated

Discrete Optimization

24

Wide range of problems LPs with additional integer constraints Knapsack & capital budgeting Set packing, covering, and partitioning Traveling salesman and routing

How do we solve these problems? Much more difficult that the

corresponding continuous problems

Exponential Growth

25

0

20

40

60

80

100

120

140

1 2 3 4 5 6 7

Number of variables

Num

ber o

f sol

utio

ns

kk22klog k

2k

k2

Relaxations

26

Discrete problems are hard Relax them to an easier problem Our original Swedish Steel formulation was a

relaxation of the real problem Can always relax a zero-one problem by

allowing the variable to take any value on the interval [0,1]

Continuous Relaxations

27

LP relaxation or continuous relaxation is when a model with discrete variables is assume to have only continuous variables

If a constraint relaxation is infeasible, the original model is infeasible as well

Relationship with Relaxed Solution

28

The optimal value of the relaxed problem is an upper/lower bound for the original maximization/minimization model

Feasible solutionsin relaxed model

Feasible solutionsin original model

Optimum

Final Relationship

29

If an optimal solution to the relaxed model is feasible for the original model, it is also optimal for the original model.

Feasible solutionsin relaxed model

Feasible solutionsin original model

Optimum for the relaxed model

Further Bound

30

Have that the optimal value of the relaxed problem is an upper/lower bound for the original maximization/minimization model

The objective value of any integer feasible solution to a maximization/minimization problem is a lower/upper bound on the integer optimal value

Rounding Example

31

integer and 0,4232503150s.t.

64.0max

21

21

21

21

xxxx

xxxx

Solve LP relaxation for (x1,x2)=(376/193,950/193)

Integer Programming (IP) solution is (x1,x2)=(5,0)

More Efficient Enumeration

32

Looking at every solution takes prohibitively long

Can we somehow account for every solution without actually looking at every solution?

Would like to be able to eliminate a bunch of solutions without evaluating them!

Branch and bound

Using Relaxations

33

If a relaxation of a candidate problem is infeasible that branch can be fathomed

If optimal solution of a relaxed candidate problem is worse than incumbent then fathom branch

If optimal solution of a relaxed candidate problem is feasible for full candidate then fathom branch and update candidate if necessary

Knapsack Model

34

Intuitive idea: what is the most valuable collection of items that can be fit into a backpack?

Capital Budgeting

35

Multidimensional knapsack problems are often called capital budgeting problems

Idea: select collection of projects, investments, etc, so that the value is maximized (subject to some resource constraints)

Assignment Problems

36

Assignment problems deal with optimal pairing or matching of objects in two distinct sets

Decision variable

Let A be the set of allowed assignments and cij be the cost of assigning i to j.

otherwise0

toassigned is if1 jixij

Traveling Salesman Problem (TSP)

37

Ames

Fort Dodge

BooneCarrollMarshalltown

West Des Moines

Waterloo

What is the shortest route,starting in Ames, that visitseach city exactly ones?

Solving TSP

38

We can use branch-and-bound to get an exact solution to the TSP problem

As always, the key to implementing branch-and-bound is to relax the problem so that we can easily solve the relaxed problem, but we still get good bounds

How can we relax the TSP?

Single Machine Sequencing

39

A set of jobs Find the best order in which to sequence the

jobs Solution

Tabu search Branch-and-bound

Neighborhood/Local Search

40

Find an initial solution Loop:

Look at “neighbors” of current solution Select one of those neighbors Decide if to move to selected solution Check stopping criterion

Random Search Methods

41

Tabu Search Maintain a tabu list of solution changes

A move made entered at top of tabu list Fixed length (5-9) Neighbors restricted to solutions not requiring a tabu

move