mdp 631 industrial operations research lecture 6 mdp 631 industrial operations research lecture 6...
TRANSCRIPT
MDP 631Industrial Operations Research
Lecture 6
Integer Programming
Today’s lecture
• Integer programming
• Branch-and-bound algorithm for MIP
• Binary decision variables and mathematical modeling of logical relationships
• Using branch-and-bound algorithm for solving BIP
Integer Programming
• Integer programming (IP) models are mathematical programming models that contain integer variables.
• Integer programming appears in a wide range of applications.
• In some cases the use of integer decision variables is more realistic than using a continuous variable.
• Examples of this include the number of workers, the number of cars to be produced and the number of courses to be taken …etc.
Why Integer Programs?
• Advantages of restricting variables to take on integer values:
– More realistic
– More flexibility (e.g. modeling logical constraints)
• Disadvantages:
– More difficult to model
– Can be much more difficult to solve
Types of IP Models
• An IP in which all variables are required to be integers is called a pure integer programming (PIP) problem.
• An IP in which only some of the variables are required to be integers is called a mixed integer programming (MIP) problem.
• An MIP that is represented by linear objective function and constraints is called mixed integer linear programming (MILP) problem.
• An integer programming problem in which all the variables must be 0 or 1 is called a 0-1 IP or binary IP (BIP).
• The LP obtained by omitting all integer or 0-1 constraints on variables is called LP relaxation of the IP.
How IPs are defined
• For integer programs: just add constraints that say that all variables that are considered integer are integer.
– In writing the model:
xi ≥ 0 and integer for all i S
– In LINGO:@GIN( <variable name>);
• For binary (0-1) decision variables:
– We write
xj {0,1} or 0 xj 1 and xj integer for all i S
– In LINGO@BIN( <variable name>);
Example: All-Integer LP
• Consider the following all-integer linear program:
Max. 3x1 + 2x2
s.t. 3x1 + x2 < 9
x1 + 3x2 < 7
- x1 + x2 < 1
x1, x2 > 0 and integer
Example: All-Integer LP
• LP Relaxation
Solving the problem as a linear program ignoring the integer constraints, the optimal solution to the linear program gives fractional values for both x1 and x2. From the graph on the next slide, we see that the optimal solution to the linear program is:
x1 = 2.5, x2 = 1.5, z = 10.5
Example: All-Integer LP
• LP Relaxation
LP Optimal (2.5, 1.5)
Max 3x1 + 2x2
-x1 + x2 < 1
x2
x1
3x1 + x2 < 9
1
3
2
5
4
1 2 3 4 5 6
x1 + 3x2 < 7
Example: All-Integer LP
ILP Optimal (3, 0)
Max 3x1 + 2x2
-x1 + x2 < 1
x2
x1
3x1 + x2 < 9
1
3
2
5
4
x1 + 3x2 < 7
1 2 3 4 5 6
Example: All-Integer LP
• Complete Enumeration of Feasible ILP Solutions
There are eight feasible integer solutions to this problem:
x1 x2 z
1. 0 0 0
2. 1 0 3
3. 2 0 6
4. 3 0 9 optimal solution
5. 0 1 2
6. 1 1 5
7. 2 1 8
8. 1 2 7
Solution of IPs
• Because we can’t just round off the solution to the LP, solving an IP is not just a matter of solving the LP relaxation.
• In fact, no algorithm exists for solving IPs that is as simple and robust as is the simplex algorithm for solving LPs.
• Branch-And-Bound (BAB or B&B) is commonly used for solving IPs and MIPs.
The Branch-And-Bound Algorithm
Basic Concepts of branch-and-boundThe basic concept underlying the branch-and-bound technique
is to divide and conquer.
Since the original “large” problem is hard to solve directly,
it is divided into smaller and smaller subproblems
until these subproblems can be conquered.
The dividing (branching) is done by partitioning
the entire set of feasible solutions into smaller and smaller subsets.
The conquering (fathoming) is done partially by
(i) giving a bound for the best solution in the subset;
(ii) discarding the subset if the bound indicates that
it can’t contain an optimal solution.
These three basic steps – branching, bounding, and fathoming –
are illustrated on the following example.
Example of Branch-and-Bound
Max Z = 5x1 + 8x2
s.t. x1 + x2 6
5x1 + 9x2 45
x1 , x2 ≥ 0 integer
x1 + x2 = 6
5x1 + 9x2 = 45
0
1
2
3
4
5
6
1 2 3 4 5 6 7 8
(2.25, 3.75)
Z=41.25
Z=20
Utilizing the information about the optimal solution of the LP-relaxation
➢ Fact: If LP-relaxation has integral optimal
solution x*, then x* is optimal for IP too.
▪ In our case, (x1, x2) = (2.25, 3.75) is the optimal
solution of the LP-relaxation. But, unfortunately,
it is not integral.
• The optimal value is 41.25 .
➢ Fact: OPT(LP-relaxation) ≥ OPT(IP)
(for maximization problems)
That is, the optimal value of the LP-
relaxation is an upper bound
for the optimal value of the integer program.
• In our case, 41.25 is an upper bound for
OPT(IP).
Max Z = 5x1 + 8x2
s.t. x1 + x2 6
5x1 + 9x2 45
x1 , x2 ≥ 0 integer
x1 + x2 = 6
5x1 + 9x2 = 45
0
1
2
3
4
5
6
1 2 3 4 5 6 7 8
(2.25, 3.75)
Z=41.25
Z=20
Branching step• In an attempt to find out more
about the location of the IP’s optimal solution,
partition the feasible region of the LP-relaxation.
• Choose a variable that is fractional in the optimal solution to the LP-relaxation – say, x2 . Observe that every feasible IP point must have either x2 3 or x2 ≥ 4 .
Max Z = 5x1 + 8x2
s.t. x1 + x2 6
5x1 + 9x2 45
x1 , x2 ≥ 0 integer
x1 + x2 = 6
5x1 + 9x2 = 45
0
1
2
3
4
5
6
1 2 3 4 5 6 7 8
(2.25, 3.75)
Z=41.25
Z=20
Branching step
• With this in mind, branch on the variable x2 to create the following two subproblems:Subproblem 1 Subproblem 2Max Z = 5x1 + 8x2 Max Z = 5x1 + 8x2
s.t. x1 + x2 6 s.t. x1 + x2 65x1 + 9x2 45 5x1 + 9x2 45x2 3 x2 ≥ 4 x1 , x2 ≥ 0 x1 , x2 ≥ 0
• Solve both subproblems(note that the original optimal solution (2.25, 3.75) can’t recur)
Max Z = 5x1 + 8x2
s.t. x1 + x2 6
5x1 + 9x2 45
x1 , x2 ≥ 0 integer
x1 + x2 = 6
5x1 + 9x2 = 45
0
1
2
3
4
5
6
1 2 3 4 5 6 7 8
(2.25, 3.75)
Z=41.25
Z=20
Branching step (graphically)
Subproblem 1: Opt. solution (3,3) with value 39
Subproblem 2: Opt. solution (1.8,4) with value 41
0
1
2
3
4
5
1 2 3 4 5 6 7 8
Z=20
Subproblem 1
Subproblem 2
(1.8, 4)
(3, 3)
Z=39
Z=41
Solution tree
For each subproblem, we record
• the restriction that creates the subproblem
• the optimal LP solution
• the LP optimum value
The optimal solution for Subproblem 1 is integral: (3, 3).
➢ If further branching on a subproblem will yield no useful information, then we can fathom (dismiss) the subproblem.
In our case, we can fathom Subproblem 1 because its solution is integral.
➢ The best integer solution found so far is stored as incumbent. The value of the incumbent is denoted by Z*.
In our case, the first incumbent is (3, 3), and Z*=39.
➢ Z* is a lower bound for OPT(IP): OPT(IP) ≥ Z* .
In our case, OPT(IP) ≥ 39. The upper bound is 41: OPT(IP) 41.
All
(2.25, 3.75)
Z=41.25
S1: x2 3
(3, 3)
Z=39
S2: x2 ≥ 4
(1.8, 4)
Z=41
int.
Next branching step (graphically)
- Fathom Subproblem 1.
- Branch Subproblem 2 on x1 :
Subproblem 3: New restriction is x1 1.
Opt. solution (1, 4.44) with value 40.55
Subproblem 4: New restriction is x1 ≥ 2.
The subproblem is infeasible
0
1
2
3
4
5
1 2 3 4 5 6 7 8
Z=20
Subpr. 3Subpr. 4
Z=40.55
(1, 4.44)
Solution tree (cont.)
➢ If a subproblem is infeasible, then it is fathomed.
In our case, Subproblem 4 is infeasible; fathom it.
➢ The upper bound for OPT(IP) is updated: 39 OPT(IP) 40.55 .
➢ Next branch Subproblem 3 on x2 .
(Note that the branching variable might recur).
All
(2.25, 3.75)
Z=41.25
S1: x2 3
(3, 3)
Z=39
S2: x2 ≥ 4
(1.8, 4)
Z=41
int.
S3: x1 1
(1, 4.44)
Z=40.55
S4: x1 ≥ 2
infeasible
Next branching step (graphically)
Branch Subproblem 3 on x2 :
Subproblem 5: New restriction is x2 4.
Feasible region:
the segment joining (0,4) and (1,4)
Opt. solution (1, 4) with value 37
Subproblem 6: New restriction is x2 ≥ 5.
Feasible region is just one point: (0, 5)
Opt. solution (0, 5) with value 40
0
1
2
3
4
5
1 2 3 4 5 6 7 8
Z=20
(1, 4)
(0, 5)
Solution tree (final)
➢ If the optimal value of a subproblem is Z*, then it is fathomed.• In our case, Subproblem 5 is fathomed because 37 39 = Z*.
➢ If a subproblem has integral optimal solution x*,
and its value > Z*, then x* replaces the current incumbent.• In our case, Subproblem 5 has integral optimal solution, and its value
40>39=Z*. Thus, (0,5) is the new incumbent, and new Z*=40.
➢ If there are no unfathomed subproblems left, then the current incumbent is an optimal solution for (IP). • In our case, (0, 5) is an optimal solution with optimal value 40.
All
(2.25, 3.75)
Z=41.25
S1: x2 3
(3, 3)
Z=39
S2: x2 ≥ 4
(1.8, 4)
Z=41
int.
S3: x1 1
(1, 4.44)
Z=40.55
S4: x1 ≥ 2
infeasible
S5: x2 4
(1, 4)
Z=37
S6: x2 ≥ 5
(0, 5)
Z=40
int.
int.
Binary Decision Variables and Mathematical Modeling of Logical
Relationships
Using binary decision variables
• Binary (or 0-1) decision variables are quite helpful in building mathematical models that contain logical constraints and relationships.
• This type of IP models appears in many applications of what is called combinatorial optimization problems.
Indicator variables
• Indicator variables are binary variables used to control the value of a continuous variable or the status of a constraint containing continuous variables in an LP model.
Controlling variables
• in this case we have a continuous decision variable x 0. Another binary variable y is added such that the following conditions hold:
y = 1 → x > 0 (1)
y = 0 → x = 0 (2)
• Since y is a binary variable that takes only two values 0 or 1, then we can say that the first condition is equivalent to:
x > 0 → y = 1 (1’)
similarly condition (2) is equivalent to:
x = 0 → y = 0 (2’)
Controlling variables
• Conditions (1) and (2) can be considered by adding the following two constraints to the original LP model:
x M y (3)
x m y (4)
where M and m are upper and lower bounds on the continuous variable x respectively.
• Note: constraint (4) can be removed from the LP model if the objective function takes the form of minimizing the following function: cx x + cy y.
Controlling constraints
• in this case we have a constraint taking the form and we want to represent the situation in
which this inequality holds or does not hold. To do that we introduce a binary variable y where the following relationships or conditions exist:
y = 0 → (5)
y = 1 → (6)
Qxqi
ii
Qxqi
ii
i i
i
q x Q
Controlling constraints
• Similar to the case of controlling a single continuous variable, the following conditions are direct results of conditions (5) and (6) respectively.
→ y = 0 (5’)
→ y = 1 (6’)
• To represent conditions (5) and (6) in the original LP model, we introduce the following constraints:
(7)
(8)
where m and M are lower and upper bounds on the difference , and is a very small number ( < 1)
Qxqi
ii
Qxqi
ii
QMMyxqi
ii ++
+−− Qymxqi
ii )(
Qxqi
ii −
Logical conditions and zero-one variables
• Logical constraints exist in many applications. They are composed of logical terms such as if A happens then B should happen, or if A or not B happen then C should happen and so on.
• Simple logical statements are constructed of constants or sentences such as A, B, and C, and connectives such as and, or and so on.
Logical relationships
Symbol Meaning
And
Or
Not
→ Implies
If and only if
Truth table
The following truth table represents the meaning for each operator:
A B A A B A B A → B
= A B
A B
F F T F F T T
F T T F T T F
T F F F T F F
T T F T T T T
Using binary variables
• Binary variables can be easily used to represent any logical statement.
• Since logical statements can be interpreted using truth tables into either true or false, binary variables having two values 0 or 1 can be used in the same manner as 0 can be used to represent the false state and 1 to represent the true state.
Using binary variables
• If the binary variables xA and xB are used to represent the logical sentences A and B, the following list shows the logical statement and its equivalent representation using binary variables.
Logical statement Equivalent binary formula
A B xA + xB
A B xA xB
A 1 - xA
A→ B xA - xB 0
A B xA - xB = 0
Example of logical constraints
• In a manufacturing problem the following logical constraint exists. If either of products A or B or both are manufactured, then at least one of the products C, D, or E must also be manufactured.
• In logical terms this constraint is represented as follows:
(A B) → (C D E)
Example of logical constraints
• The equivalent mathematical model for this constraint can be obtained by first assigning binary decision variable for each product: xA, xB, xC, xD, xE.
• These binary variables can be incorporated into the mathematical model the same way the indicator variables are used.
Example of logical constraints
• Then to use a mathematical modeling equivalent of the logical statement (A B) → (C D E), we need to break it step by step.
• We have a single implication statement which is the main part of the logical statement, then we need to introduce two binary variables for each part of this statement.
• The left part will have the binary variable xL and the right part will have the binary variable xR. Then the implication statement is represented mathematically as follows:
xL - xR 0
Example of logical constraints
• Now xL should be linked with the binary variables that represent the left hand side of the implication statement. To do that we use the following constraints:
xL xA + xB 2 xL
Similarly
xR xC + xD + xE 3 xR
Special ordered sets of variables
• The concept of special order sets is specially developed to represent common situations in linear programming. They are two types:– Special ordered sets of type 1 (SOS1): is an ordered set of variables
(continuous or integer) within which exactly one variable must be non-zero.
– Special ordered sets of type 2 (SOS2): is an ordered set of variables within which at most two consecutive variables are non-zero.
• The conditions arising in the SOSs can be easily represented mathematically by using binary variables. The following examples illustrate how this can be done
Special ordered sets of variables (Example 1)
• A depot can be sited in any location A, B, C, D or E. only one depot can be built.
• Here, we use binary variables xA, xB, xC, xD and xE to represent the condition that the depot is built in any one of them.
• Then the SOS in this case is the set of binary variables.
• The following constraint needs to be added to represent the condition that only one location will be assigned to build the depot:
xA + xB + xC + xD + xE = 1
Special ordered sets of variables (Example 2)
• If we have an SOS2 of the variables xA, xB, xC, xD and xE the SOS2 conditions can be represented by using additional decision variables zi where i = 1 to 4 (the number of variables-1). The following constraints will be added:
xA z1
xB z1 + z2
xC z2 + z3
xD z3 + z4
xE z4
And z1 + z2 + z3 + z4 = 1
Illustrative examples on using logical constraints
Example 1: Capital Budgeting IP
• Stockco is considering four investments.
• Each investment:
– Yields a determined NPV.
– Requires a certain cash flow at the present time.
• Currently Stockco has $14,000 available for investment.
• Formulate an IP whose solution will tell Stockco how to maximize the NPV obtained from the four investments.
Example 1: Solution
• Begin by defining a variable for each decision that Stockcomust make.
• The NPV obtained by Stockco is:Total NPV obtained by Stockco = 16x1 + 22x2 + 12x3 + 8x4
• Stockco’s objective function is:max z = 16x1 + 22x2 + 12x3 +8x4
• Stockco faces the constraint that at most $14,000 can be invested.
• Stockco’s 0-1 IP is:
max z = 16x1 + 22x2 + 12x3 +8x4
s.t. 5x1 + 7x2 + 4x3 +3x4 ≤ 14xj = 0 or 1 (j = 1,2,3,4)
How to Model “logical” Constraints
• At most 2 stocks are selected.
• Exactly 3 stocks are selected.
• Either of stock 2 or stock 3 can be selected but not both.
• If stock 1 is selected, then so is stock 2.
• If stock 3 is selected, then stock 4 is not selected.
Some Simple Constraints
• If we have a bound on the number of stocks selected we just insert a constraint like:
x1 + x2 + x3 + x4 2
“and” or “or” Constraints
• Exactly 3 stocks are selected can be modeled using:x1 + x2 + x3 + x4 = 3
• Either of stock 2 or stock 3 can be selected but not both, can be modeled using:x2 + x3 ≤ 1
Basic if-then Constraints
• If stock 1 is selected then stock 2 is also selected can be modelled using:
x1 x2
• If stock 3 is selected then stock 4 is not allowed to be selected can be modelled using:
x3 1-x4
Either-or constraints
• We know how to represent either-or forms in terms of the decision variables.
• Now we want to know how to deal with either-or situations when it is a pair of constraints that must be decided between.
• If we start with two constraints f(x) 0 and g(x) 0 and exactly one of them must be true we add a binary variable ywhich forces one of them to be true.
• We then add the two constraints to the formulation using:f(x) My
g(x) M(1-y)
Either-or constraints
• If the binary variable y=1 then the two constraints are equivalent to:f(x) Mg(x) 0
• And if y=0 then:f(x) 0g(x) M
• Thus we ensure that exactly one of the two original constraints is true by forcing both of the new constraints to be true.
Example 2: Dorian Autos
• In this example the only real concern we have is that Dorian Autos must produce a minimum number of 1000 cars if it produces any at all.
• If we define xi to be the number of cars produced of type i, then we know that either xi=0 or xi≥1000.
• Putting these into the form required by either-or constraints would give:xi 0 and 1000-xi 0
Ex 6 Dorian Autos cont’d
• The two constraints:xi Miyi
1000 – xi Mi(1-yi)
• The value of Mi needs to be selected and may be different for each type of car.
Solving BIP models using Branch and Bound Algorithm
Example of Binary IP
All the decision variables have the binary form
Because the last two decisions represent mutually exclusive alternatives (the
company wants at most one new warehouse), we need the constraint
x1: building a factory in Los Angeles? Yes (x1=1) or No (x1=0)
x2: building a factory in San Francisco? Yes (x2=1) or No (x2=0)
x3: building a warehouse in Los Angeles? Yes (x3=1) or No (x3=0)
x4: building a warehouse in San Francisco? Yes (x4=1) or No (x4=0)
Furthermore, decisions 3 and 4 are contingent decisions, because they are
contingent on decisions 1 and 2, respectively (the company would consider
building a warehouse in a city only if a new factory also were going there). Thus,
in the case of decision 3, we require that x3 = 0 if x1 = 0. This restriction on x3
(when x1 = 0) imposed by adding the constraint
Similarly, the requirement that x4 = 0 if x2 = 0 is imposed by adding the constraint
The complete BIP model for this problem is
California Manufacturing Co. Example
Branching
Original problem
Solution Tree
The variable used to do this branching
at any iteration by assigning values to
the variable is called the branching
variable
Bounding
LP relaxation of the whole problem
LP relaxation of subproblem 1 LP relaxation of subproblem 2
Fathoming
Three cases where a subproblem is conquered (fathomed).
(1) A subproblem is conquered if its LP relaxation has an integer optimal solution
(2) A subproblem is conquered if it is inferior to the current incumbent.
Since Z*=9, there is no reason to consider further any subproblem whose
bound ≤ 9, since such a subproblem cannot have a feasible solution better than
the incumbent. Stated more generally, a subproblem is fathomed whenever its
(3) If the simplex method finds that a subproblem’s LP relaxation has no
feasible solutions, then the subproblem itself must have no feasible solutions, so
it can be dismissed (fathomed).
Using the BIP Branch-and Bround Algorithm to
Solve the California Manufacturing Co. Example
(1) Initiliazation
Set Z*= − ∞. Solve the relaxation of the whole problem by the
simplex method. The optimal solution of the relaxation is
1)The bound of the whole problem is less than Z*.
2)The relaxation of the whole problem has feasible solution.
3)The optimal solution includes a noninteger value of x1.
So the whole problem can not be fathomed and should be divided
(branched) into subproblems.
(2) Iteration 1Subproblem 1 with x1=0. The optimal solution of its relaxation is (0,1,0,1) with Z=9.
The optimal solution is integer, which is the best feasible solution found so far. So this
integer solution with Z*=9 is stored as the first incumbent. Since the optimal solution of
its relaxation is integer, subproblem 1 is fathomed
Subproblem 2 with x1=1. Because subproblem 2 is not fathomed, it should be divided
into subproblems.
Subproblem 1
Subproblem 2
(3) Iteration 2The only remaining subproblem corresponds to the x1=1 node, so we shall branch from
this node to create the two new subproblems.
Subproblem 4 with x1=1, x2=1.
Subproblem 3 with x1=1, x2=0. LP relaxation of subproblem 3
LP relaxation of subproblem 4
Bound for subproblem 3 :
Bound for subproblem 4 :
Subproblem 1
Subproblem 2
Subproblem 3
Subproblem 4
(4) Iteration 3So far, the algorithm has created 4 subproblems. Subproblem 1 has been fathomed, and
subproblem 2 has been replaced by subproblems 3 and 4, but these last two remain under
consideration. Because they were created simultaneously, but subproblem 4 has the larger
bound, the next branching is done from subproblem 4, which creates the following new
subproblems
Subproblem 5 with x1=1, x2=1, x3=0.
Subproblem 6 with x1=1, x2=1, x3=1.
LP relaxation of subproblem 5 :
LP relaxation of subproblem 6 :
No feasible solutions
Bound for subproblem 5 : Z≤ 16
Subproblem 1
Subproblem 2
Subproblem 3
Subproblem 4
Subproblem 5
Subproblem 6
(5) Iteration 4The subproblems 3 and 5 corresponding to nodes (1,0) and (1,1,0) remain under
consideration. Since subproblem 5 was created most recently, so it is selected for
branching.
Since x4 is the last variable, fixing its value at either 0 or 1 actually creates a single
solution rather than subproblems. These single solutions are
(1,1,0,0) with Z=14 is better than
the incumbent with Z*=9, so
(1,1,0,0) with Z*=14 becomes the
new incumbent.
Because a new incumbent has been
found, we reapply fathoming test 1
with the new incumbent to the only
remaining subproblem 3.
Subproblem3:
There are no remaining (unfathomed) subproblems. Therefore, the optimality test
indicates that the current incumbent is optimal.
Subproblem 1
Subproblem 2
Subproblem 3
Subproblem 4
Subproblem 5
Subproblem 6
Last Comments on IP models
• There are often multiple ways of modeling the same combinatorial optimization problem.
• Solvers for integer programs are extremelysensitive to the formulation. (not true for LPs)