elen0445-1 - microgrids [2mm] introduction to mathematical

ELEN0445-1 - Microgrids

Introduction to mathematical programming (v1)

Bertrand Cornelusse

ULiege - Institut Montefiore

2017

ELEN0445-1 1/43

Lecture overview

I The goal of this lecture is to introduce the concept of mathematicalprogramming, and in particular of linear and mixed integer programming.

I We also quickly review solution methods for these problems.

I These tools can then be used for real-time dispatch, operational planning,or sizing of a microgrid.

ELEN0445-1 2/43

Outline

Mathematical programming

Linear programming

Integer and Mixed-Integer programming

Modeling techniques

Cutting planes

Branch and bound

ELEN0445-1 3/43


Mathematical programming is a field of applied mathematics that deals withthe solution of optimization problems.

More precisely, it provides a framework and solution methods for computing thedecisions of an optimization problem, given an objective function to minimizeor maximize, and (optionally) constraints on the decisions variables.

Mathematical programming relies on a model of the problem to solve.

There is a great variety of mathematical programming problem types,depending on the characteristics of the objective function and of theconstraints, and of the restrictions that apply to variables.

ELEN0445-1 4/43

Categories of mathematical programs

General mathematical program

A general mathematical programcan be stated as follows:

min f (x)

s.t. g(x) ≤ 0

Ax = 0

x ∈ X

It is very hard to solve, especiallywhen

I objective and constraints arenon-linear or even worsenon-convex

I variables are discrete

Linear program

min cT x

s.t. Ax = b

x ∈ Rn+

Easy to solve even for largeproblems.

Mixed-Integer Linear program

min cT x

s.t. Ax = b

x ∈ Rn1+ ∗ Z

n2+

Hard problem, but feasible formoderately sized instances.

ELEN0445-1 5/43

Outline


Linear programming


Modeling techniques

Cutting planes

Branch and bound

ELEN0445-1 6/43

Linear programming

If the objective is linear and the constraints are linear, we talk about linearprogramming (LP) or linear optimization.

LP in standard form

min cT x

s.t. Ax = b

x ∈ Rn+

DefinitionA polyhedron is a set {x ∈ Rn|Ax ≥ b}

A set of the form Ax ≤ b is also a polyhedron.A set {x ∈ Rn|Ax = b, x ≥ 0} is a polyhedron in standard form.

ELEN0445-1 7/43

Graphic representationWe can represent a problem in two dimensions graphically.Example:

max x1+2x2 (1)

−x1+2x2 ≤ 1 (2)

−x1+ x2 ≤ 0 (3)

4x1+3x2 ≤ 12 (4)

x1, x2 ≥ 0 (5)

ELEN0445-1 8/43

Graphic representation

max x1+2x2 (1)

−x1+2x2 ≤ 1 (2)

−x1+ x2 ≤ 0 (3)

4x1+3x2 ≤ 12 (4)

x1, x2 ≥ 0 (5)

1

(3)

(2)

(4)

x

x2

ELEN0445-1 8/43


max x1+2x2 (1)

−x1+2x2 ≤ 1 (2)

−x1+ x2 ≤ 0 (3)

4x1+3x2 ≤ 12 (4)

x1, x2 ≥ 0 (5)

��

��

(3)

(2)

(4)

x

x2

1ELEN0445-1 8/43


max x1+2x2 (1)

−x1+2x2 ≤ 1 (2)

−x1+ x2 ≤ 0 (3)

4x1+3x2 ≤ 12 (4)

x1, x2 ≥ 0 (5)

��

��

(3)

(2)

(4)

x

x2

1+2xx1 =22

ELEN0445-1 8/43


max x1+2x2 (1)

−x1+2x2 ≤ 1 (2)

−x1+ x2 ≤ 0 (3)

4x1+3x2 ≤ 12 (4)

x1, x2 ≥ 0 (5)

��

��

(3)

(2)

(4)

x

x2

1+2xx1 =42

ELEN0445-1 8/43


max x1+2x2 (1)

−x1+2x2 ≤ 1 (2)

−x1+ x2 ≤ 0 (3)

4x1+3x2 ≤ 12 (4)

x1, x2 ≥ 0 (5)

��

��

(3)

(2)

(4)

x

x2

1+2xx1 2=4.8

ELEN0445-1 8/43

Extreme points and vertices

DefinitionLet P be a polyhedron. A point x ∈ P is an extreme point of P if there do notexist two points y , z ∈ P such that x is a convex combination of y and z .

DefinitionLet P be a polyhedron. A point x ∈ P is a vertex of P if there exists c ∈ Rn

such that cT x < cT y for all y ∈ P and y 6= x .

ELEN0445-1 9/43

Degenerate cases

In the example we had a unique solution at a vertex of the polyhedron.Some degenerate cases can lead to different solutions.

ELEN0445-1 10/43

Degenerate cases


min x1 + x2

s.t. − x1 + x2 ≤ 1

x1, x2 ≥ 0

x

x2

1

ELEN0445-1 10/43

Degenerate cases


min x1

s.t. − x1 + x2 ≤ 1

x1, x2 ≥ 0

1x

x2

ELEN0445-1 10/43

Degenerate cases


max − x1 + x2

s.t. − x1 + x2 ≤ 1

x1, x2 ≥ 0

1x

x2

ELEN0445-1 10/43

Degenerate cases


max x1 + x2

s.t. − x1 + x2 ≤ 1

x1, x2 ≥ 0

x

x2

1

5

12

3

4

ELEN0445-1 10/43

Degenerate cases


max x1 + 2x2

s.t. − x1 + x2 ≤ 1

− x1 + x2 ≥ 2

x1, x2 ≥ 0x2

1x

ELEN0445-1 10/43

Bases of a polyhedron

We subdivide the equalities and inequalities into three categories:

aTi x ≥ bi i ∈ M≥

aTi x ≤ bi i ∈ M≤

aTi x = bi i ∈ M=

DefinitionLet x be a point satisfying aTi x = bi for some i ∈ M≥,M≤ or M=. Theconstraint i is said to be active or tight.

ELEN0445-1 11/43

Bases of a polyhedron

DefinitionLet P be a polyhedron and let x ∈ Rn.

(a) x is a basic solution ifI all equalities (i ∈ M=) are activeI among the active constraints, there are n linearly independent

(b) if x is a basic solution that satisfies all constraints, then x is a feasiblebasic solution.

TheoremLet P be a polyhedron and let x ∈ P. The three following statements areequivalent.

(i) x is a vertex

(ii) x is an extreme point

(iii) x is a basic feasible solution

ELEN0445-1 12/43

Linear programming algorithms

There are two main types of algorithms used in practice.

Simplex methods

I moves from one vertex (extreme point) of the feasible domain to anotheruntil objective stops decreasing

I very efficient in practice but can be exponential on some special problems

I can keep information of one solution to quickly compute a solution to aperturbed problem (useful in a B&B setting), dual simplex, ...

Interior point methods

I iteratively penalizes the objective with a function of constraints, to forcesuccessive points to lie within the feasible domain

I polynomial time, very efficient especially for large sparse systems

I but no extremal solution hence crossover required in a B&B setting

ELEN0445-1 13/43

More advanced topics

I Duality

I Shadow prices

I Complementary slackness

I Sensitivity analyses

I ...

ELEN0445-1 14/43

Outline


Linear programming


Modeling techniques

Cutting planes

Branch and bound

ELEN0445-1 15/43

Modeling a discrete problem

Consider the problem

min c(x)

s.t. f (x) ≤ b

g(x) = 0

x ∈ X .

When

I c, f , g are linear

I X = Zn+

This is called Integer (Linear) Programming (IP).Remarks:

I Mixed Integer Programming (MIP) when some variables have acontinuous domain.

I If c is nonlinear, very little has been done (except the quadratic case)

I If f or g is nonlinear: even less

ELEN0445-1 16/43

Why is that so complicated?

I After all, there is a finite number of solutions

I In particular, n! possible permutations

I Imagine we can check 1012 possibilities per secondThat is already a pretty amazing machine!

10! 0 sec20! 28 days30! 8400 billion years40! 5 quadrillions times the age of the Earth. . .

I Let us not dare to continue. . .

ELEN0445-1 17/43

Example: Uncapacitated Lot Sizing (ULS)

You are producing bikes and youknow in advance the demand dt forT time steps ahead. Producing attime t induces a fixed cost ft , andthe variable cost per bike producedis ct . There is no storage cost.Formulate the MIP that allows youto compute the production plan thatminimizes the total production costto satisfy the demand.

Formulation as a MIP:

minx,y

T∑t=1

ftxt +T∑t=1

ctyt

s.t.t∑

u=1

yu ≥t∑

u=1

du, ∀t ∈ T

yt ≤

(T∑

u=t

du

)xt , ∀t ∈ T

yt ≥ 0, ∀t ∈ Txt ∈ {0, 1}, ∀t ∈ T

ELEN0445-1 18/43

Outline


Linear programming


Modeling techniques

Cutting planes

Branch and bound

ELEN0445-1 19/43

Binary choice

A choice between 2 alternatives is modeled through a 0, 1-variable.

Example: the knapsack problem

maximizen∑

i=1

cixi

subject ton∑

i=1

aixi ≤ b

xi ∈ {0, 1} for all i = 1, . . . , n.

ELEN0445-1 20/43

Forcing constraints

If decision A is made then decision B must be made as well.

x = 1 if decision A is taken y = 1 if decision B is taken

x = 0 otherwise y = 0 otherwise

The constraint readsx ≤ y

Example: Facility Location problem

I m clients (i = 1, 2, . . . ,m) to satisfy (demand = 1)

I n potential locations for facilities (j = 1, 2, . . . , n)

I Can serve client i from facility j only if facility j is open:

xij ≤ yj

I xij fraction of demand of client i served by facility jI yj ∈ {0, 1}, 1 if facility is open.

ELEN0445-1 21/43

Restricted range of values

Suppose we want to formulate x ∈ {a1, a2, . . . , am}.

We introduce m binary variables yj .

x =m∑j=1

ajyj ,m∑j=1

yj = 1, yj ∈ {0, 1}

ELEN0445-1 22/43

Disjunctive constraints

I Consider a variable x ≥ 0,

I we want to model that either aT1 x ≥ b1 or aT2 x ≥ b2,

I and a1 ≥ 0, a2 ≥ 0.

We introduce a variable y ∈ {0, 1} that represents whether constraint 1(y = 1) or constraint 2 is satisfied, and replace both constraints by

aT1 x ≥ yb1 and aT2 x ≥ (1− y)b2.

x1

x2

and

aT1 x ≥ b1

aT2 x ≥ b2

x1

x2

y = 1

x1

x2

y = 0

Exercise:

I extend to N disjunctive constraints;

I what if you want that exactly k of the N constraints satisfiedsimultaneously?

ELEN0445-1 23/43

Disjunctive constraints (...)

I Now consider 0 ≤ x ≤ U,

I we want to express either aT1 x ≤ b1 or aT2 x ≤ b2,

I without restriction on a1 nor a2.

Again, introduce variable y ∈ {0, 1} and parameter M defined as

M = maxm,0≤x≤U

{m : m ≥ aTi x − bi , i = 1, 2

},

thenaT1 x − b1 ≤ My and aT2 x − b2 ≤ M(1− y).

Example: scheduling problem

I Two tasks, starting time t1, t2 ≥ 0, duration P1,P2 ≥ 0

I either task 1 is performed before task 2, or the opposite

I hence either t1 ≥ t2 + P2, or t2 ≥ t1 + P1

ELEN0445-1 24/43

Arbitrary piecewise linear cost functions

f (x)

x

(X1,F1)

(X2,F2)

(X3,F3)

(X4,F4)Introduce variables bi ∈ {0, 1} suchthat

bi = 1 if x ∈ [Xi ,Xi+1]

bi = 0 if x 6∈ [Xi ,Xi+1]

Formulation 1:

I∑

i bi = 1

I xi ≤ bi

I f =∑

i (Fibi + xi (Fi+1 − Fi ))

Formulation 2:

I∑

i bi = 1

I λi ≤ bi−1 + bi , i = 2, . . . , n − 1

I λ1 ≤ b1, λn ≤ bn−1

I∑

i λi = 1

I f =∑

i λiFi

Exercise: what if f (x) is convex and we want to solve minx{f (x) s.t. x ∈ X}?

ELEN0445-1 25/43

Outline


Linear programming


Modeling techniques

Cutting planes

Branch and bound

ELEN0445-1 26/43

The linear (continuous) relaxation

DefinitionGiven the Mixed Interger Program:

zMIP = min cT x + dT y

s.t. Ax + By = b

x , y ≥ 0

y ∈ Rn

x ∈ Zn,

its linear relaxation is defined as

zLP = min cT x + dT y

s.t. Ax + By = b

x , y ≥ 0

y ∈ Rn

x ∈ Rn.

I The linear relaxation gives important information about the optimal valueof an integer program:

zLP ≤ zMIP ,

I hence, it is easy to obtain a lower bound (solving the relaxation is “easy”),

I but in general hard to obtain an integer solution from the solution of therelaxation without elaborated techniques.

ELEN0445-1 27/43

Relaxation strength

I Alternative formulations of a problem may lead to different linearrelaxations.

I If the formulation is ideal, that is, the LP relaxation defines the convexhull of the feasible set of the Integer Program, we need nothing else thanLinear Programming algorithms. This often requires an exponentialnumber of constraints.

I Here, we consider a different approach: automatically derive validinequalities from the original constraints of the model in order toapproximate the convex hull of the feasible points of the IP.

ELEN0445-1 28/43

Illustration

max 5x1 + 11x2 (6)

s.t. x1 ≤ 6 (7)

x1 − 3x2 ≥ 1 (8)

3x1 + 2x2 ≤ 19 (9)

x1, x2 ∈ Z+ (10)

x2

x10

(15) (16)

(14)

(13)

ELEN0445-1 29/43

Illustration

(14)− (15)

3and (17)⇒ x2 ≤ 1 (11)

(11) + (16)

3and (17)⇒ x1 + x2 ≤ 6

(12)

x2

x10

(15) (16)

(14)

(11)(12)

(13)

ELEN0445-1 30/43

Valid inequalities

DefinitionLet P ⊆ Rn. An inequality

∑nj=1 ajxj ≤ b is valid if it is satisfied by all points

x ∈ P.

Typically,

I we want to derive valid inequalities for the set of integral solutions

I and at the same time cut off some part of the linear programmingrelaxation.

ELEN0445-1 31/43

Outline


Linear programming


Modeling techniques

Cutting planes

Branch and bound

ELEN0445-1 32/43

Introduction

I Here, we see an algorithm for solving to optimality an Integer Programwhen its formulation is not ideal: branch-and-bound.

I Other algorithms such as cutting-planes, are almost always used inconjunction with branch-and-bound (leading to the well knownbranch-and-cut algorithm).

ELEN0445-1 33/43

Consider the following problem

max 5x1 + 11x2 (13)

s.t. x1 ≤ 6 (14)

x1 − 3x2 ≥ 1 (15)

3x1 + 2x2 ≤ 19 (16)

x1, x2 ∈ Z+ (17)

ELEN0445-1 34/43

Geometrical view

x2

x10

(15) (16)

(14)

(13)

ELEN0445-1 35/43

What information does the LP relaxation yield?

I Objective: z?,0 ≈ 42.82

I Solution: x?,0 ≈ (5.36, 1.45)

x2

x10

(15) (16)

(14)

I Idea: enumerate, i.e. iteratively restrict the domain of x , but using theinformation of the linear relaxation.

I The enumeration yields a search tree.

I The root node of this tree is called the root relaxation (node 0 in thesequel).

ELEN0445-1 36/43

Search tree

0

... ...

x1 ≤ C

... ...

x1 ≤ D

x1 ≤ A

... ...

x2 ≤ E

... ...

x2 ≤ F

x1 ≥ B

Remark: A, B, . . . , F ∈ Z+

ELEN0445-1 37/43

Use information of the relaxation to ...

I decide on which variables to branch

I set the thresholds

I prune parts of the search tree

I ...

0

... ...

x1 ≤ C

... ...

x1 ≤ D

x1 ≤ A

... ...

x2 ≤ E

... ...

x2 ≤ F

x1 ≥ B

ELEN0445-1 38/43

Back to our example, branch on x1: x1 ≤ 5 (Node 1)

I z?,1 ≈ 39.67

I x?,1 = (5, 4/3)

x2

x10

(15) (16)

(14)

ELEN0445-1 39/43

Node 2: from node 1, branch on x2: x2 ≤ 1

I z?,2 = 36

I x?,2 = (5, 1)

x2

x10

(15) (16)

(14)

ELEN0445-1 40/43

Node 4: the second alternative from the root node: x1 ≥ 6

I z?,4 = 35.5

I x?,4 = (6, 1/2)

x2

x10

(15) (16)

(14)

ELEN0445-1 41/43

Branch and bound tree

1

0

2 3

4

x2 ≤ 1 x2 ≥ 2

x1 ≤ 5 x1 ≥ 6Remark: node index = order ofexploration ! = order of creation.

NodeNodes left Objective

Nb integer infeasible variables

Best Integer

Best Bound Gap Decision

0 2 42,82 2 / 42,82 Infinite Branch1 2 39,67 1 / 42,82 Infinite Branch2 2 36 0 36 42,82 15,93% prune by optimality3 1 "-infinity" / 36 42,82 15,93% prune by infeasibility4 0 35,5 1 36 35,5 0,00% prune by bound

ELEN0445-1 42/43

Remarks

I Opportunities to prune the search:I by bound,I by optimality,I by infeasibility

I Need of a good primal bound in the beginningI Different strategies for the node selection:

I depth-first-search (good to find quickly primal solutions)I breadth-first-search (good to increase the dual bound)

I Different strategies for variable selection:I Most fractional variable or least fractional variableI Take advantage of the history of branchingI Look ahead for best improvement in the bound: strong branching

ELEN0445-1 43/43

elen0445-1 - microgrids [2mm] introduction to mathematical

Documents