math models of or: extreme points and basic feasible...
TRANSCRIPT
Math Models of OR:
Extreme Points and Basic Feasible Solutions
John E. Mitchell
Department of Mathematical Sciences
RPI, Troy, NY 12180 USA
September 2018
Mitchell Extreme Points and Basic Feasible Solutions 1 / 26
Introduction
Outline
1 Introduction
2 Let x̄ be a basic feasible solution, show it is an extreme point
3 Let x̄ be an extreme point, show it is a basic feasible solutionExample
The general case
Mitchell Extreme Points and Basic Feasible Solutions 2 / 26
Introduction
Extreme points
Recall that if a standard form linear optimization problem
minx2IRn cT xsubject to Ax = b
x � 0
has an optimal solution then it has an optimal solution that is an
extreme point.
Definition
Let P be the feasible set of a linear program. A point x̄ 2 P is anextreme point of P if it is not on the line segment joining two otherpoints in P.
Mitchell Extreme Points and Basic Feasible Solutions 3 / 26
Q:extreme
Introduction
Basic feasible solutions
The simplex algorithm works with canonical form tableaus and moves
from basic feasible solution to adjacent basic feasible solution.
The basic feasible solution corresponding to a canonical form
tableau is obtained by setting the nonbasic variables equal to zero and
then finding the unique solution for the basic variables.
Mitchell Extreme Points and Basic Feasible Solutions 4 / 26
x , x - x3 4 4 X i " " Basic sequencet¥¥¥¥¥¥ ' s i n '115=4,4=2,x ,s 9
9X ,e x , 2×4=+620
Introduction
The relationship
We’ve seen graphically that basic feasible solutions correspond to
extreme points. We prove the following theorem to make this formal:
Theorem
Let x̄ be a feasible solution to a standard form linear optimizationproblem. The point x̄ is a basic feasible solution if and only if it is anextreme point of the feasible region of the linear optimization problem.
The theorem is an “if and only if” theorem, so we need to show two
directions.
Mitchell Extreme Points and Basic Feasible Solutions 5 / 26
Introduction
For example
x1
x2
0
1
2
2 4 6
x1 +
3x2 =
6, x3 =
0x1 = 3,x4 = 0
x2 = 2, x5 = 0
optimal contour
�x1 � 2x2 = �5
minx2IR2 �x1 � 2x2
subject to x1 + 3x2 6
x1 3
x2 2
x1, x2 � 0
Mitchell Extreme Points and Basic Feasible Solutions 6 / 26
t .EE:
Let x̄ be a basic feasible solution, show it is an extreme point
Outline
1 Introduction
2 Let x̄ be a basic feasible solution, show it is an extreme point
3 Let x̄ be an extreme point, show it is a basic feasible solutionExample
The general case
Mitchell Extreme Points and Basic Feasible Solutions 7 / 26
Let x̄ be a basic feasible solution, show it is an extreme point
Assume x̄ is not an extreme point and derive a
contradiction
Since x̄ is assumed to not be extreme, it must be the midpoint between
two other distinct feasible points x (1) and x (2), so x̄ = 1
2x (1) + 1
2x (2).
This means that for each component j = 1, . . . , n, we have
x̄j =1
2x (1)
j +1
2x (2)
j .
x (1) = (1, 4)
x (2) = (11, 2)
line segment
x̄ = (6, 3)
Mitchell Extreme Points and Basic Feasible Solutions 8 / 26
S h o u t i B F SAssume also I i s a B f f , g e t a contradiction
Let x̄ be a basic feasible solution, show it is an extreme point
Nonbasic components of x̄We look in particular at nonbasic components of x̄ , so x̄j = 0.
For these components, we have
x̄j = 1
2x (1)
j + 1
2x (2)
j
= 0 for � 0 since � 0 since
nonbasic x (1) feasible x (2) feasible
components
It follows that we must have x (1)j = x (2)
j = 0 for all the nonbasic
components of x̄ .
But once the nonbasic components are fixed, the values of the basic
variables are then uniquely determined: they are equal to the
corresponding entries in b.
So we must have x (1) = x (2) = x̄ . So the points x (1) and x (2) are notdistinct, so x̄ is actually an extreme point.
Mitchell Extreme Points and Basic Feasible Solutions 9 / 26
I f i n o t extreme, couldhave
I = tax'" t # x " , for example.( i )
,X• ¥ N/ o ×
redefine
×")← {× '"+1×4)
Now I = ± (new +"1/+11×14)
E j =#f" + taxi"(8)i t (4)+ 'il't)
F¥tx÷÷¥l¥÷÷dSince x I", xs'"7 0 ,
m u s t have ×g'"ex ,"= O
Let x̄ be an extreme point, show it is a basic feasible solution
Outline
1 Introduction
2 Let x̄ be a basic feasible solution, show it is an extreme point
3 Let x̄ be an extreme point, show it is a basic feasible solutionExample
The general case
Mitchell Extreme Points and Basic Feasible Solutions 10 / 26
Let x̄ be an extreme point, show it is a basic feasible solution
Introduction to proof
This direction is harder to prove, and the proof needs some linear
algebra.
We prove it by contradiction, so we assume x̄ is not a BFS and showit is not an extreme point.
To make the proof a little simpler, we assume x̄ has exactly m positivecomponents; the proof can be extended to handle the general case.
Mitchell Extreme Points and Basic Feasible Solutions 11 / 26
h o w tochartacterire?
A i s m e n
Let x̄ be an extreme point, show it is a basic feasible solution Example
Outline
1 Introduction
2 Let x̄ be a basic feasible solution, show it is an extreme point
3 Let x̄ be an extreme point, show it is a basic feasible solutionExample
The general case
Mitchell Extreme Points and Basic Feasible Solutions 12 / 26
Let x̄ be an extreme point, show it is a basic feasible solution Example
An example
We first work an example before abstracting the proof.
minx2IR6 3x1 + 2x2 + x3
subject to x1 + x2 + x3 + x4 = 6
2x1 + x3 + x5 = 4
�x1 + x2 + x6 = 2
x1, . . . , x6 � 0
The point x̄ = (1, 3, 2, 0, 0, 0) is feasible. Is it a BFS? The tableau is
x1 x2 x3 x4 x5 x6
0 3 2 1 0 0 0
6 1 1 1 1 0 0
4 2 0 1 0 1 0
2 �1 1� 0 0 0 1
Mitchell Extreme Points and Basic Feasible Solutions 13 / 26
Let x̄ be an extreme point, show it is a basic feasible solution Example
Two pivots
x1 x2 x3 x4 x5 x6
0 3 2 1 0 0 0
6 1 1 1 1 0 0
4 2 0 1 0 1 0
2 �1 1� 0 0 0 1
R0 � 2R3,R1 � R3
�!
x1 x2 x3 x4 x5 x6
�4 5 0 1 0 0 �2
4 2 0 1� 1 0 �1
4 2 0 1 0 1 0
2 �1 1 0 0 0 1
R0 � R1,R2 � R1
�!
x1 x2 x3 x4 x5 x6
�6 3 0 0 �1 0 �3
4 2 0 1 1 0 �1
0 0 0 0 �1 1 �1
2 �1 1 0 0 0 1Mitchell Extreme Points and Basic Feasible Solutions 14 / 26
or:±÷i÷÷÷:c u l1 = - cu l 2
+ 2 (co13)
&
Let x̄ be an extreme point, show it is a basic feasible solution Example
We don’t have a BFS
x1 x2 x3 x4 x5 x6
�6 3 0 0 �1 0 �3
4 2 0 1 1 0 �1
0 0 0 0 �1 1 �1
2 �1 1 0 0 0 1
The second constraint does not involve x1, x2, or x3.
So our given feasible solution is not a bfs.
Mitchell Extreme Points and Basic Feasible Solutions 15 / 26
Let x̄ be an extreme point, show it is a basic feasible solution Example
A little linear algebra
Equivalently, we can say that this means the original first threecolumns of A are linearly dependent.
In fact, notice that
2
41 1 1
2 0 1
�1 1 0
3
5
2
41
1
�2
3
5 =
2
40
0
0
3
5 .
Because of this equality, we have a direction we can move in and still
maintain feasibility.
Mitchell Extreme Points and Basic Feasible Solutions 16 / 26
F td , I - d both feasible
I = ÷ ( I td)+ t h - d )
7- D =Eli'?'""}= (1,312,...) ✓i e - e d@D = (1,1,-2,490)feqifl.ee#-xee%Ad=0
Let x̄ be an extreme point, show it is a basic feasible solution Example
The direction
Let d = (1, 1,�2, 0, 0, 0)T 2 IR6. Then
x̂ := x̄ + td
satisfies Ax̂ = b for any t , positive or negative, since Ax̄ = b and
Ad = 0.
Further, provided �1 t 1, we get x̂ � 0.
Mitchell Extreme Points and Basic Feasible Solutions 17 / 26
Let x̄ be an extreme point, show it is a basic feasible solution Example
Two new feasible points
Want to show x̄ is not extreme.
Let’s take the two new feasible points x (1) and x (2) with t = 1 and
t = �1, respectively (the x4, x5, and x6 components of d , x (1), x (2) and
x̄ are all zero, so we don’t write them out explicitly):
x (1) = x̄ + d =
2
41
3
2
3
5 +
2
41
1
�2
3
5 =
2
42
4
0
3
5
x (2) = x̄ � d =
2
41
3
2
3
5 �
2
41
1
�2
3
5 =
2
40
2
4
3
5
Mitchell Extreme Points and Basic Feasible Solutions 18 / 26
Let x̄ be an extreme point, show it is a basic feasible solution Example
x̄ is not extreme
Notice that
1
2x (1) +
1
2x (2) =
1
2
2
42
4
0
3
5 +1
2
2
40
2
4
3
5 =
2
41
3
2
3
5 = x̄ .
So x̄ is not an extreme point.
Mitchell Extreme Points and Basic Feasible Solutions 19 / 26
Let x̄ be an extreme point, show it is a basic feasible solution Example
Graph this example
x1
x2
x3
0
4
2 62
6
2
6 x1 + x2 + x3 = 6, x4 = 0
Mitchell Extreme Points and Basic Feasible Solutions 20 / 26
Let x̄ be an extreme point, show it is a basic feasible solution Example
Graph this example
x1
x2
x3
0
4
2 62
6
2
6 x1 + x2 + x3 = 6, x4 = 0
�x1 + x2 = 2, x6 = 0
Mitchell Extreme Points and Basic Feasible Solutions 20 / 26
Let x̄ be an extreme point, show it is a basic feasible solution Example
Graph this example
x1
x2
x3
0
4
2 62
6
2
6 x1 + x2 + x3 = 6, x4 = 0
�x1 + x2 = 2, x6 = 0
Mitchell Extreme Points and Basic Feasible Solutions 20 / 26
Let x̄ be an extreme point, show it is a basic feasible solution Example
Graph this example
x1
x2
x3
0
4
2 62
6
2
6 x1 + x2 + x3 = 6, x4 = 0
�x1 + x2 = 2, x6 = 0
2x1 + x3 = 4, x5 = 0
Mitchell Extreme Points and Basic Feasible Solutions 20 / 26
Let x̄ be an extreme point, show it is a basic feasible solution Example
Graph this example
x1
x2
x3
0
4
2 62
6
2
6 x1 + x2 + x3 = 6, x4 = 0
�x1 + x2 = 2, x6 = 0
2x1 + x3 = 4, x5 = 0
portion offeasible region with
x4 = x5 = x6 = 0
(2, 4, 0) = x (1)
(0, 2, 4) = x (2)
(1, 3, 2) = x̄
Mitchell Extreme Points and Basic Feasible Solutions 20 / 26
Let x̄ be an extreme point, show it is a basic feasible solution Example
Linear algebra characterization of BFS
The example illustrates the following result:
Let x̄ be a feasible solution to a standard form linear program with
m ⇥ n constraint matrix A with rank equal to m.
The point x̄ is a basic feasible solution if and only if the columns of
A corresponding to the positive components of x̄ are linearly
independent.
Mitchell Extreme Points and Basic Feasible Solutions 21 / 26
Let x̄ be an extreme point, show it is a basic feasible solution The general case
Outline
1 Introduction
2 Let x̄ be a basic feasible solution, show it is an extreme point
3 Let x̄ be an extreme point, show it is a basic feasible solutionExample
The general case
Mitchell Extreme Points and Basic Feasible Solutions 22 / 26
Let x̄ be an extreme point, show it is a basic feasible solution The general case
Return to the general case
Recall, we assume x̄ is not a basic feasible solution and try to derive a
contradiction.
Without loss of generality, we can assume the m positive components
of x̄ are the first m components, by renumbering the components if
necessary. So we have
x̄j
⇢> 0 for j = 1, . . . ,m= 0 for j = m + 1, . . . , n
Mitchell Extreme Points and Basic Feasible Solutions 23 / 26
- - -
Let x̄ be an extreme point, show it is a basic feasible solution The general case
Row reducing A
We can write the constraint matrix A as
A =
2
64 B|{z}m columns
N|{z}(n�m) columns
3
75
If x̄ was a basic feasible solution, we would be able to use
elementary row operations to turn this into a canonical form, so
A = [B N] �!hI N̂
i
for some m ⇥ (n � m) matrix N̂, where I is the m ⇥ m identity matrix.
Mitchell Extreme Points and Basic Feasible Solutions 24 / 26
positive component,
✓o l ' x
✓' {Imponenti
a f I
Let x̄ be an extreme point, show it is a basic feasible solution The general case
But we don’t have a BFS
Since x̄ is not a basic feasible solution, this row reduction cannot be
possible. That means the columns of B are linearly dependent, which
is equivalent to stating that
there exists a vector dB 2 IRm with BdB = 0,where dB has at least one nonzero component.
We can extend dB out to a vector d 2 IRn by appending zeroes:
let dj =
⇢dB
j for j = 1, . . . ,m0 for j = m + 1, . . . , n
so d =
dB
0
�m components
(n � m) components
Mitchell Extreme Points and Basic Feasible Solutions 25 / 26
Let x̄ be an extreme point, show it is a basic feasible solution The general case
Moving in our direction
Notice that
Ad = [B N]
dB
0
�= BdB = 0,
so for any scalar t we have
A(x̄ + td) = Ax̄ + tAd = b + 0 = b.
Further, since dj = 0 if x̄j = 0, we have x̄ + td � 0 for t sufficiently
close to zero (positive or negative).
Thus, choose t̄ > 0 so that both x̄ + t̄d and x̄ � t̄d are feasible.
Then notice that x̄ is the midpoint of these two feasible points:
x̄ =1
2
�x̄ + t̄d
�+
1
2
�x̄ � t̄d
�.
Thus, x̄ is not an extreme point. Thus, we’ve proved the
contrapositive, so we do indeed have the result that if x̄ is an extreme
point then it is a basic feasible solution.
Mitchell Extreme Points and Basic Feasible Solutions 26 / 26