practical mathematical models of optimization … · 2 chapter 0. introduction 0.1 what is this...

PRACTICAL MATHEMATICALMODELS OF OPTIMIZATION

PROBLEMSUSING GOOGLE OR-TOOLS

SERGE KRUK

DECEMBER 14, 2015

Contents

Contents i

0 Introduction 10.1 What is this book about? . . . . . . . . . . . . . . . . . . . . . . 20.2 Features of the text . . . . . . . . . . . . . . . . . . . . . . . . . 30.3 Getting our feet wet: Bacterial coexistence . . . . . . . . . . . . 4

1 Linear continuous models (LP) 151.1 Diet problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171.2 Blending problems . . . . . . . . . . . . . . . . . . . . . . . . . 241.3 Project management . . . . . . . . . . . . . . . . . . . . . . . . 301.4 Multi-stage (multi-period) models . . . . . . . . . . . . . . . . 35

2 Linear pseudo-integer models (LP) 432.1 Maximum flow problem . . . . . . . . . . . . . . . . . . . . . . 452.2 Minimum cost flow problem . . . . . . . . . . . . . . . . . . . . 512.3 Transshipment problem . . . . . . . . . . . . . . . . . . . . . . 562.4 Shortest paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612.5 Piecewise linear . . . . . . . . . . . . . . . . . . . . . . . . . . . 702.6 Curve fitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

3 Pure linear integer models (IP) 853.1 Minimum Set Cover . . . . . . . . . . . . . . . . . . . . . . . . . 873.2 Set Packing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 933.3 Bin Packing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 963.4 TSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

i

ii CONTENTS

4 Mixed linear models (MIP) 1154.1 Facility Location . . . . . . . . . . . . . . . . . . . . . . . . . . . 1174.2 Multi-Commodity Flow . . . . . . . . . . . . . . . . . . . . . . 1224.3 Staffing level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1294.4 Cutting Stock . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1354.5 Non-convex trickery . . . . . . . . . . . . . . . . . . . . . . . . 1454.6 Staff Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . 1594.7 Sports Timetabling . . . . . . . . . . . . . . . . . . . . . . . . . 1674.8 Puzzles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176Quick reference for OR-Tools MPSolver in Python . . . . . . . . . . 189

Chapter 0

Introduction

1

2 CHAPTER 0. INTRODUCTION

0.1 What is this book about?This book is an introduction to the art and science of optimization mod-elling1 more precisely mathematical models of optimization problems. An opti-mization problem is almost any problem that is, or can be, formulated as aquestion starting with “What is the best . . . ”? For instance:

• What is the best route to get from home to work?

• What is the best way to produce cars to maximize profit?

• What is the best way to carry groceries home, paper or plastic?

• Which is the best school for my kid?

• Which is the best fuel to use in rocket boosters?

• What is the best placement of transistors on a chip?

• What is the best NBA schedule?

The number of possible examples is infinite. Note that the questions, asstated above, are rather vague, in the sense that they can be interpreted ina multitude of ways. Consider the first one: By best route do we mean thefastest, the shortest, the most pleasant to ride, the less bumpy, the one usingless fuel? Moreover the question is incomplete: Are we walking, riding,driving, snowboarding? Are we alone or dragging a screaming two-year-old along?

To help us formulate solutions to optimization problems, optimizers2

have established a frame into which we mold the questions; we call that amodel. The most crucial aspect of a model is that it has an objective andit has constraints. Roughly, the objective is what we want; the constraints,what obstacles are on our way. If we can reformulate the question to clearlyidentify both the objective and the constraints, we are closer to a model.

Let us consider in more detail the “best route” problem but with an eyeto clarify objective and constraints. We could formulate it as:

1Creating a model is more art than science; solving a model is more science than art.2I will use the term “optimizers” to name the mathematicians, theoreticians and prac-

tioners, who, since the nineteen-fiftees, have worked in the fields of Linear Programming(LP) and Integer Programming (IP). There are other who could make valid claims to themoniker, chefly among them researchers in Constraint Programmming, but my focus willbe mostly in LP and IP models, hence my restricted definition.

0.2. FEATURES OF THE TEXT 3

Given a map of the city, my home address and the address of thedaycare of my two-year-old son, what is the best route to takeon my bike to bring him to daycare as fast as possible?

The goal is to find, among all the solutions that satisfy the requirements,that is, they are paths following either streets or bike lanes (the constraints),one path that minimizes the time it takes to get there (the objective).

Objectives are always quantities we want to maximize or minimize (time,distance, money, surface area, . . . ). Though we will see examples where wewant to maximize something and minimize something else; this is easily ac-commodated. Sometimes there are no objectives. We say that the problem isone of feasibility, i.e. we are looking for any solution satisfying the require-ments. From the point of view of the modeller, the difference is minimal.Especially since, in most practical cases, a feasibility model is usually a firststep. After noticing a solution one usually wants to optimize somethingand the model is modified to include an objective function.

0.2 Features of the textAs this text is an introduction, I do not expect the reader to be already wellversed in the art of modelling. I will start at the beginning, assuming onlythat the reader understands what is a variable (both in the mathematicalsense and in the programming sense), an equation, an inequality and afunction. I will also assume that the reader knows some programming lan-guage, preferably Python, though knowing any other imperative languageis enough to be able to read the Python code displayed in the text.

Note that the code in this book is an essential component. To get thefull value, the reader must, slowly and attentively, read the code. This bookis not a text of recipes described from a birds-eye view, using mathematicalnotation, with all the nitty-gritty details “left as an exercise for the reader”.This is implemented, functional, tested, optimization code that the readercan use and is encouraged to modify to fully understand. The mathematicsin the book has been reviewed by mathematicians, like any mathematicalpaper. But the code has been subjected to a much more stringent set ofreviewers with names Intel, AMD, Motorola, IBM. 3

The book is the fruit of decades of consulting and of years teachingboth an introductory modelling class (MOR242 Intro to Operation Research

3My doctoral advisor used to say “There are error-free mathematical papers”. But weonly have found an existence proof of that theorem. I will not claim that the code is error-free, but I am certain that it has fewer errors than any mathematical paper I ever wrote.


Models) and a graduate class (APM568 Mathematical Modelling in Indus-try) at Oakland University. I start at the undergraduate level and proceedup to the graduate level in terms of modelling itself, without delving muchinto the attendant theory.

• Every model is expressed in Python using Google or-tools4 and canbe executed as stated. In fact, the code presented in the book is au-tomatically extracted, executed and the output inserted into the textwithout manual intervention; even the graphs are produced automat-ically (thanks to Emacs 5 and org-mode 6).

• My intention is to help the reader become a proficient modeller, nota theoretician, therefore little of the fascinating mathematical theoryrelated to optimization is covered; it is nevertheless used, and usedprofitably, to create simple yet efficient models.

• The associated web site provides all the code presented in the bookalong with a random generator for many of the problems and varia-tions. The author uses this as a personalized homework generator. Itcan also be used as a self-guided learning tool.

http://www.practicalopt.com/

0.2.1 A note on notation

Throughout the book, I will describe algebraic models. These models can berepresented in a number of ways. I will use two. I will sketch each modelusing common mathematical notation typeset with TEX in math mode. Iwill then express the complete, detailed model in executable Python code.The reader should have no problem seeing the equivalence between theformulations. Table 0.1 illustrates some of the equivalencies.

0.3 Getting our feet wet: Bacterial coexistenceThe simplest problems are similar to those first encountered in high-school(the dreaded word problems). They are algebraic in nature; that is, can be

4https://developers.google.com/optimization/library5The one and only editor: http://www.gnu.org/software/emacs/6http://orgmode.org/

http://www.practicalopt.com/

https://developers.google.com/optimization/library

http://www.gnu.org/software/emacs/

http://orgmode.org/

0.3. GETTING OUR FEET WET: BACTERIAL COEXISTENCE 5

TABLE 0.1: Equivalence of expression in Math and Python modes.

Object Math mode Python modeScalar Variable x xVector vi v[i]Matrix Mij M[i][j]Inequality x + y ≤ 10 x+y <= 10Summation ∑9

i=0 xi sum(x[i] for i in range(10))Set definition {i2 | i ∈ [0, 1, . . . , 9]} [i**2 for i in range(10)]

TABLE 0.2: Units of food consumed by each strain of bacteria

Food Strain I Strain II Strain III AvailableA 2 1 1 1500B 1 3 2 3000C 1 2 3 5000

formulated and sometimes solved using the simple tools of elementary lin-ear algebra. We consider here one such problem to illustrate our approachto modeling and define some fundamental concepts.

A biologist will place three strains of bacteria (denoted I, II and III) in atest tube where they will feed on three different food sources (denoted A,Band C). Each day 1500 units of A, 3000 units of B and 5500 units of C will beplaced in the tube and each bacterium consumes a certain number of unitsof each food per day. The Table 0.2 summarizes the relevant data.

The biologist wants to know how many bacteria, up to 1000 of eachstrain, can coexist in the test tube assuming that food is the only relevantconstraint.

How to we model this problem? All optimization and feasibility prob-lems in this book are modeled using a three step approach. We will expandon this approach as we encounter problems on increasing complexity butthe fundamental three steps remain the cornerstone of a good model.

• Identify the question to answer. This identification should take theform of a precise sentence involving either counting or valuating oneor more objects. In this case: “How many bacteria of each type A, Band C can coexist in the test tube?”. Notice that “How many bacteria?”would not not be precise enough because we are not interested in thetotal bacteria count, but rather in the count of each strain. Formulatinga precise question is often the hardest part.


Once we have this precise question, we assign a variable to each ofthe objects to count. Here, we could use x0, x1 and x2. These are tradi-tionally known as decision variables. This expression is a misnomer inour first example but reflects the origins of optimization problems inmilitary logistics where the decision variables were indeed represen-tative of quantities under the control of the modeller, or maybe of hiscommanding officer.

• Identify all requirement and translate them into constraints. Theconstraints, as we will see throughout the book, can take on a multi-tude of forms. In this simple problem, they are algebraic, linear in-equalities. It is often best to write down each requirement in a precisesentence before translating it into a constraint. For the bacterial case,the requirements, in words, are:

– The number of units of food A consumed by all strains of bacteriais 1500 units.

– The number of units of food B consumed by all strains of bacteriais 3000 units.

– The number of units of food C consumed by all strains of bacteriais 5000 units.

Note that a statement starting with “The amount of . . .” may not beprecise enough. In our simple case, there are no specified units butthere could be. For instance the amount consumed could be stated ingrams while the availability is in kilograms. This happens often andis the cause of many a model going awry.

Yet, even with our seemingly precise statements, there is an ambiguityleft to consider. It is one of the main contribution of a good modelerto highlight ambiguity and clarify problem statements. Here, do wemean that the bacteria will consume exactly the amounts stated, orthat they will consume at most the amounts stated7? We will assumethat “at most” is the proper form of the requirement, both because it ismore interesting and, in a sense, subsumes the “equal” question. We

7This seemingly trivial change, from “exactly equal” to “at most” represents more than2000 years of mathematical development in solution techniques. We have known how tosolve the “equal” form since ancient Babylonians (though it is known today as “Gaussianelimination”) and we teach it in high school, but we only discovered how to solve the “atmost” form in the twentieth century.


will then translate these requirements into algebraic constraints basedon our decision variables.

Let us consider food A. Strain I eats 2 units per bacterium. Each ofstrain II and III eats 1 unit per bacterium. Since we have x0 bacteriaof strain I, x1 bacteria of strain II and x2 bacteria of strain III, the totalnumber of units consumed will be the sum of our decision variables.Hence we obtain

2x0 + x1 + x2 ≤ 1500 (1)

Had we decided that “equal to” was the proper constraint, we wouldreplace the inequality by an equality.

For food B, strain I consumes 1 unit per bacterium; strain II consumes3 units and strain III consumes 2 units. They will collectively consumex0 + 3x1 + 2x2 and we obtain the constraint

x0 + 3x1 + 2x2 ≤ 3000 (2)

The last constraint is obtained similarly.

x0 + 2x1 + 3x2 ≤ 5000 (3)

• Identify the objective to optimize. The objective is, in the case of anoptimization problem, what we want to maximize (or minimize). Inthe case of a feasibility problem, there is no objective but in practice,most feasibility problems are really optimization problems that havebeen incompletely formulated.

Since the problem is stated as “How many bacteria of each strain cancoexist?”, a possible, even likely reading is that we want the maximumnumber of bacteria. (Clearly, the mimimum number is zero and un-interesting.) In terms of our decision variables, we want to maximizetheir sum. We obtain the objective

max x0 + x1 + x2 (4)

At this point we have a model! Not the model, but a model; a simple,clear and precise algebraic model that has a solution, a solution which an-swers our original question. The next task, unless one is a theoretician withno interest in practical answers, –but then why would you be reading thisbook?– is to run it.


As we will do for every model in this book, we need to translate themathematical expressions above ((1)-(4)) into a form digestible by one ofthe many solvers available.

Over the years, optimizers have developed a number of specialized mod-elling languages and solvers. Here is a short list of the better known ones:

• Modelling languages

– AMPL (www.ampl.com)

– GAMS (gams.com)

– GMPL (http://en.wikibooks.org/wiki/GLPK/GMPL_(MathProg))

– Minizinc (http://www.minizinc.org/)

– OPL (http://www-01.ibm.com/software/info/ilog/)

– ZIMPL (http://zimpl.zib.de/)

• Solvers

– CBC (http://www.coin-or.org/)

– CLP (http://www.coin-or.org/Clp/)

– CPLEX (http://www-01.ibm.com/software/info/ilog/)

– ECLiPSe (http://eclipseclp.org/)

– Gecode (http://www.gecode.org/)

– GLOP (https://developers.google.com/optimization/lp/glop)

– GLPK (http://www.gnu.org/software/glpk/)

– Gurobi (http://www.gurobi.com/)

– SCIP (http://scip.zib.de/)

We should maintain a distinction between modelling languages, formalconstructions with specific vocabulary and grammars; and solvers, softwarepackages that can read in models expressed in certain languages and writeout the solutions. Though in some cases, this distinction is blurry.

As a modeller, one creates a model (in language X) which is then fed toa solver (solver Y). This can happen because solver Y knows how to parselanguage X or because there is a translator between language X and anotherlanguage, say Z, which the solver understands. This, over the years, hasbeen the cause of much irritation (“What? You mean that I have to re-writemy model to use your solver?”).


To make matters worse, these languages and solvers are not equivalent.Each has its strengths and weaknesses, its areas of specialization. Afteryears of writing models in all the languages above, and then some, mypreference today is to eschew specialized languages and to use a generalpurpose programming language, for instance Python, along with a libraryinterfacing with multiple solvers. Throughout this book, I will use Google’sOperations Research Tools (or-tools) a very well structured and easy to uselibrary.

The or-tools library is comprehensive. It offers the best interface I haveever used to access multiple linear and integer solvers (MPSolver). It alsohas special-purpose code for network flow problems as well as a very effec-tive constraint programming library. In this text, I will display only a verysmall fraction of this cornucopia of optimization tools.

One of the many advantages of using a general purpose language likePython is that we can not only do the modelling part, but we can also easilyinsert these models into a larger application, maybe a web or a phone app.We can also easily present the solutions in a clear format. We have all thepower of a complete language at our disposal. True, the specialized mod-elling languages sometimes allow more concise model expression. But inmy experience, they all, at one point or another, hit a wall and the modelleris forced to write kludgy glue to connect a model to the rest of the applica-tion. Moreover, writing or-tools models in Python can be such a joy8. Thewhole bacterial coexistence model at is shown at Code 1.

CODE 1: Bacteria solver1 from linear_solver import pywraplp2 def solve_bacteria():3 t = ’Bacterial␣coexistence’4 s = pywraplp.Solver(t,pywraplp.Solver.GLOP_LINEAR_PROGRAMMING)5 x = [s.NumVar(0, 1000,’x[%i]’ % i) for i in range(3)]6 pop = s.NumVar(0,3000,’pop’)7 s.Add(2*x[0] + x[1] + x[2] <= 1500)8 s.Add(x[0] + 3*x[1] + 2*x[2] <= 3000)9 s.Add(x[0] + 2*x[1] + 3*x[2] <= 4000)

10 s.Add(pop == x[0] + x[1] + x[2])11 s.Maximize(pop)12 s.Solve()13 return pop.SolutionValue(),[e.SolutionValue() for e in x]

8Writing in Common Lisp would be even better, alas, there is no Lisp binding for or-tools yet. I am working on it.


Let us deconstruct the code. Line 1 loads the Python wrapper of theLinear Programming subset of or-tools. Every model we write will start thisway. The line 4 names and creates a linear programming solver (hereafternamed s), using Google’s own GLOP9 The or-tools library has interfaces toaa number of solvers. Switching to a different solver, say GNU’s GLPK10 orCoin-or CLP11 is a simple matter or modifying this line.

On line 5, we create a one-dimensional array x of three decision variablesthat can take on values between 0 and 1000. The lower bound is a physicalconstraint as we cannot have a negative number of bacteria. The upperbound is part of the problem statement as the biologist will not put morethan 1000 of each strain in the test tube. It is possible to state ranges as anycontiguous subsets of (−∞,+∞), but, as a general rule of thumb, restrictingthe range as much as possible during variable declaration tends to helpsolvers run efficiently. The third parameter of the call to NumVar is used asthe name to print if and when this variable is displayed, for instance, indebugging a model. We will have little use for this feature as we prefer towrite bug-free models.

The constraints on lines 7 to 9 are direct translations of the mathematicalexpressions (1)-(3). The order of the terms is irrelevant. In contrast to somerestrictive modelling languages, we could have written line 7 as

1500>=x[0]+x[2]+x[1]

or

x[0]+x[1]+x[2]-1500<=0

or any other equivalent algebraic expression.At line, 6, we declare an auxiliary variable, pop. Though there is no such

distinction in the modelling language, this is not a decision variable butrather a helpful device to model the problem. We use this auxiliary on line10 where we add an equation that does not constrain the model in any way.It is simply defining the auxiliary variable pop to be the sum of our decisionvariables. This allows us to express the objective easily and, possibly, tohelp display the solution.

We have the objective function on line 11, a translation of (4). The func-tion choices are, unsurprisingly, either s.Maximize or s.Minimize with, forparameter, a linear expression in terms of the variables declared previously.We used

9https://developers.google.com/optimization/lp/glop10https://www.gnu.org/software/glpk/11https://projects.coin-or.org/Clp

https://developers.google.com/optimization/lp/glop

https://www.gnu.org/software/glpk/

https://projects.coin-or.org/Clp


s.Maximize(pop)

We could have written

s.Maximize(x[0]+x[1]+x[2])

We then call on the solver at line 12 to do its job. This is where all thecomputational work gets done; work that we will not describe. The inter-ested reader can search for “Simplex method” and “Interior-point methods”to learn all about the fascinating theory behind the solution methods of lin-ear optimization models. To understand the simplex method, one needsonly high-school algebra. To understand interior-point methods requiressomewhat more mathematical background.

For some models, solvers may complete their work in a fraction of asecond; for others, it may take hours. Moreover not all solvers will havethe same runtime behavior. Model A may run faster than model B on solverX while it may be exactly the reverse on solver Y. One more advantage orusing the or-tools library is that we can try out another solver by changingone line.

We should, if this code were meant for production and the problem non-trivial, check the return value to ensure that the solver found an optimalsolution. It may have aborted because of a model error, or because it ranout of time or memory, or for some other reason. But for our simple firstexample, we will forgo good engineering practice in the name of simplicityof exposition.

We return, on line 13, both the optimal objective function value held invariable pop, as well as the optimal values of the decision variables (not allthe associated object attributes carried by those variables).

On more complex models, we may post-process the decision variablesto return something simpler and more meaningful to the caller. We will seea good example of this when we solve the shortest path problem in Section2.4. The general approach we encourage is to create models that can beused without any knowledge of the internals of or-tools. The modeller isresponsible for the creation of the model, but once the model is created andvalidated, it should leave the hands of its creator for those of the domainexpert who originally formulated the problem.

CODE 2: How to call the bacteria solverfrom bacteria import solve_bacteriapop,x=solve_bacteria()T=[[’Strain’, ’Quantity’]]


TABLE 0.3: Solution to the bacteria problem

Strain QuantityI 100.0II 300.0III 1000.0Total 1400.0

for i in range(3):T.append([[’I’,’II’,’III’][i], round(x[i])])

T.append([’Total’, pop])print(T)

If the reader executes the bacteria code, for example as in Code 2 (andthe reader is encouraged to do so) he will see a result much like Table 0.3.Notice that we can look at the solution and see that it does indeed satisfythe constraints. By substituting the solution into (1)-(3), we obtain

2(100.0) + 300.0 + 1000.0 = 1500 ≤ 1500100.0 + 3(300.0) + 2(1000.0) = 3000 ≤ 3000100.0 + 2(300.0) + 3(1000.0) = 3700 < 5000

Notice that the first two inequalities are satisfied with equality. In the jar-gon of optimization, such inequalities are tight or active. The last one is saidto be slack or inactive. In a certain sense, we could delete it from the prob-lem and nothing would change. (The reader can try this and other modi-fications. The code is available in the additional material under the namebacteria.py.)

In summary, the steps to construct and run a model are:

• Formulate the question precisely.

• Define the decision variables by identifying what is required to an-swer the question.

• Possibly define auxiliary variables to help simplify the statements ofconstraints or of the objective function. They can also help in the anal-ysis and the presentation of the solution.


• Translate each constraint into an algebraic equality or inequality in-volving directly the decision variables or indirectly, through the aux-iliary variables.

• Construct the objective function as some quantity that should be min-imized or maximized.

• Run the model using an appropriate solver.

• Display the solution in an appropriate manner.

• Validate the results: Does the solution correctly satisfy the constraints?Is the solution meaningful and implementable? If so, declare that weare done; if not, consider the necessary modifications to the model.

The rest of the text will construct models of increasing complexity, illus-trating and expanding the points above.

Chapter 1

Linear continuous models (LP)

15

16 CHAPTER 1. LINEAR CONTINUOUS MODELS (LP)

At the dawn of optimization (the fifties), the state-of-the-art was definedby linear optimization models and the simplex method, the only reasonablyefficient algorithm known at the time to solve such models. When I startedstudying this subject, one repeatedly heard from multiple sources that over70% of the cpu cycles in the world were devoted to running various simplexcodes. Surely an exaggeration, but it is indicative of the power of linearmodels. The world is not linear, but sometimes, a linear approximation isgood enough.

More precisely, we discuss here Linear Continuous models (though ev-erybody calls them LPs for Linear Programs, implying the continuous prop-erty). Linear continuous models are the simplest to write down and the sim-plest to solve. They have been the workhorse of optimizers since Dantziginvented the Simplex method to solve them. What characterizes them isthree elements:

• All variables are continuous.

• All constraints are linear.

• The objective function is linear.

In more detail, the decision variables (say x0, . . . , xn) can take on integraland fractional values. This is appropriate when, for instance, the solution iscounting divisible objects, say pounds of flour. It is not appropriate whenthe solution is counting people unless one is looking only for an approxi-mation.

The objective function is (or can be) parameterized by a constant vectorc and expressed as

c1x1 + c2x2 + ... + cnxn

This precludes objective functions with terms of the form x2, sin(x), x1x2or |x| for instance. Though we will see later how to handle some of thesenon-linearities.

Finally, the constraints are parameterized by a matrix aij, a vector b, andcan be stated as a set of relations, for i ∈ {1, . . . , m}:

ai1xi1 + ai2xi2 + ... + ainxin ≥ bi or ≤ bi or = bi

or some equivalent algebraic form.In this chapter we consider problems where the natural, obvious formu-

lation is such a linear continuous model.

1.1. DIET PROBLEM 17

1.1 Diet problemThe canonical LP example is the diet problem. The diet problem is one ofthe first optimization problems to be studied back in the thirties and for-ties (twentieth century, not twenty-first1). The, likely apocryphal, origin ofthe problem is the US military’s desire to meet the nutritional requirementsof the field GIs while minimizing the cost of the food. One of the earlyresearchers to study this problem was George Stigler. He made an edu-cated guess of the optimal solution to the linear program using a heuris-tic method. In the fall of 1947, Jack Laderman of the Mathematical TablesProject of the National Bureau of Standards (NBS, today NIST) undertooksolving Stigler’s model with the new simplex method. The linear modelconsisted of nine equation in 77 unknowns, a huge problem for the time.Some models in this book are orders of magnitude larger and will be solvedin a minuscule fraction of the time it took the NBS people to solve the dietproblem in 1947. The increase in efficiency is partly due to the hardware,but mostly due to the software.

A generic version of the problem is:

Given a list of food with some nutritional content, each with acost, find the combination of food that will minimize cost andyet provide all the necessary nutrients.

Here is one simple version of this problem. We will let the foods be numbers0, 1, 2, 3, . . . (Imagine them to be pizza, ramen noodles, cupcakes etc. . . or,if you are of a more health-conscious bent, tofu, green peas, quinoa, etc. . . )The nutrients be N0, N1, N2, N3, . . . (Imagine them to be calories, protein,calcium, vitamin A, etc. . . ). Each has a cost per serving. In addition, toavoid eating one food all week long, let us put restrictions on the numberof servings per week.

A randomly generated instance is given in the Table 1.1.2 Each row rep-resents a food, with the nutritional content per serving, followed by theacceptable range of servings of the food and its cost per serving. Ignore thelast row and column for now. We will return to them after the model is con-structed and solved. The two rows before last represent the allowed rangeof each nutrient.

1I add this temporal precision on the odd chance that this text is still being read longafter by body has maximized its entropy.

2To encourage the reader to experiment, every model in this book is available in theadditional material, along a random instance generator.


TABLE 1.1: Example of data and solution for the diet problem.

N0 N1 N2 N3 Min Max Cost SolutionF0 606 563 665 23 7 17 9.06 17.0F1 68 821 83 72 6 27 8.42 7.47F2 28 70 916 56 1 36 9.47 6.11F3 121 429 143 38 14 26 6.97 14.0F4 60 179 818 46 9 35 4.77 35.0Min 5764 28406 48157 1642Max 15446 76946 82057 6280Sol 14775 28406 48157 3413 539.37

1.1.1 Constructing a model

What would a solution be but a list of servings of each food. Thereforethe decision variables must be one per food, representing the number ofservings. Let us name these variables f0, . . . , fn. We will assume that it isacceptable to have fractional answers. (i.e. one half serving is acceptable.)

The objective is to minimize cost. We have one cost per food (say c0, . . . , cnthese are not variables, they are data) therefore, what we want is to mini-mize the sum of all the products ci × fi. This leads to the objective function

min ∑i

ci fi

Let us tackle the constraints. We have two sets: one indicating the rangeof acceptable servings of each food (assume that the minimum of food iis li and maximum is ui) and one indicating the required nutrients range(mininimum of nutrient j is aj and maximum is bj). The simpler constraintis related to the food. Since our decision variables indicate the number ofservings of each food, we need only to box each serving count:

li ≤ fi ≤ ui (1.1)

The constraint on nutrients is a bit more involved. Consider nutrient j.How much of it will be included in the diet? Each food i may have someof it, as indicated in Table 1.1. Let us call this amount Nji (correspondingto the entry at row of food i and the column of nutrient j). To get the totalof this nutrient, we therefore need to sum over all foods the product of thefood serving and the nutrient content. For each nutrient j:

aj ≤∑i

Nji fi ≤ bj


We are done with the theory. Let us translate this into an executablemodel general enough to solve all problems of this type. We will assumethat the data is given in a 2-dimensional array called N. It has the structureof Table 1.1 without the last column and row. Each row represents a food,except that the last two rows represent the minimum and maximum re-quirement of each nutrient, represented by the columns, with the last threerepresenting the minimum, maximum and the cost of each food serving.

CODE 1.1: Diet solver1 def solve_diet(N):2 s = newSolver(’Diet’)3 nbF,nbN = len(N)-2, len(N[0])-34 FMin,FMax,FCost,NMin,NMax = nbN,nbN+1,nbN+2,nbF,nbF+15 f = [s.NumVar(N[i][FMin], N[i][FMax],’f[%i]’%i) for i in range(nbF)]6 for j in range(nbN):7 s.Add(N[NMin][j] <= s.Sum([f[i]*N[i][j] for i in range(nbF)]))8 s.Add(s.Sum([f[i]*N[i][j] for i in range(nbF)]) <= N[NMax][j])9 s.Minimize(s.Sum([f[i]*N[i][FCost] for i in range(nbF)]))

10 rc=s.Solve()11 return rc,ObjVal(s),SolVal(f)

The model uses the newSolver function to simplify the expression of thecode3 as the reader can see at Code 1.2. These, and other simplification, canbe found in my_or_tools.py.

CODE 1.2: Utility function to create an appropriate solver instancefrom linear_solver import pywraplpdef newSolver(name,integer=False):return pywraplp.Solver(name,\

pywraplp.Solver.CBC_MIXED_INTEGER_PROGRAMMING \if integer else \pywraplp.Solver.GLOP_LINEAR_PROGRAMMING)

To help the expression of the model, lines 3 - 4 give meaningful namesto the row and column indices that we will use. In line 5 we define thedecision variables, one per food, each taking values in the range [li, ui] as in(1.1). It would be correct to give a range of [0,+∞] and then add constraintsto enforce the bounds. The solver would still find the same solution; but it issimpler and good practice to limit as much as possible the range of decision

3Mostly to make the code fit a page, but also to hide some of the verbosity of the or-tools library. The authors having chosen, rightly in my opinion, meaningful, but ratherlong names for their functions.


variables. In complex models, it often dramatically improves the solutiontime.

The two-line loop starting on line 6 establishes the range on each nutri-ent as in (1.1.1). Lines 9 and following create the objective function, solvethe problem and return three numbers: the status of the solver (it should bezero), the optimal value and optimal solution. The dual role of the functionsSolVal and ObjVal (seen at Code 1.3) is simplify the results returned to thecaller and the code to read.

CODE 1.3: Utility functions to extract values from the or-tools objectsdef SolVal(x):if type(x) is not list:return 0 if x is None else x if isinstance(x,(int,float)) else x.

SolutionValue() if x.Integer() is False else int(x.SolutionValue())

elif type(x) is list:return [SolVal(e) for e in x ]

def ObjVal(x):return x.Objective().Value()

The results from executing this model are shown in the last row andcolumn of Table 1.1. The column indicates the number of servings of eachfood and the row indicates the amount of each nutrient that will be in thediet. The reader should notice that many of the food items and nutrientcounts are at their minimum required values. This is expected of such amodel since we are trying to minimize a linear cost function; the optimalsolution should push towards the boundary of the constraints as much aspossible.

The reader can experiment with this model. It is included in the addi-tional material as diet_problem.py, along with a generator of random dietproblems and a routine to display the solution in a table format similar toTable 1.1.

1.1.2 Variations

• There are a number of simple variations of this problem. For instance,instead of minimizing cost, we could be given a profit to maximize.We could also not have either the minima or the maxima in either thefoods or nutrients.


• It becomes more complex, consequently interesting, when we have, inaddition, requirements of the form: "If food 2 is used, then we musthave at least as much food 3 in the diet". Or "Nutrient 3 must beincluded in at least twice the amount as nutrient 4".

Let us consider some of these in detail. First "If food 2 is used, thenfood 3 must also be included in at least as many servings". The fol-lowing inequality ensures the required result:

f3 ≥ f2

Notice that food 3 could still be included when food 2 is not, but thatdoes not violated the requirement. And if food 2 is included, then wewill have at least as many servings of food 3. It should be clear thatthe requirement could have been stated in reverse "No more food 2than food 3." The constraint is the same.

• A requirement on the nutrients: "Nutrient 3 must be included in atleast twice the amount as nutrient 4." has a similar flavour but wenote that the amount of any given nutrient is spread among all thefood items. It may be fruitful to introduce auxiliary variables that willtally the nutrients, say nj. We then add to the model one equality pernutrient:

nj = ∑i

Nij fi

Note that these equations do not constrain the problem; their insertionis simply a helpful device to implement the requirement. We can noweasily relate the nutrient content according to the new requirement as

n3 ≥ 2n4

which we could have stated, had we not defined the variables ni, as

∑i

Ni3 fi ≥ 2 ∑i

Ni4 fi

Defining the auxiliary variables nj seems clearer. Moreover, display-ing the total of each nutrient at the end might help with the analysisor the presentation of the solution.

• A similar requirement may occur to the reader, namely "If food (nutri-ent) 3 is used then food (nutrient) 4 must not be (and vice-versa). Thismay look like a simple variation to the above but it is decidedly not


Components Availabilities CostC1 . . . Cn Min Max

ProductsP1 99 . . . 99 99 99 99. . . . . . . . . . . . . . . . . . . . .Pm 99 . . . 99 99 99 99

Demand Min 99 . . . 99 99 99 99Max 99 . . . 99 99 99 99

Price 99 . . . 99 99 99 99

TABLE 1.2: Abstract structure of product mix problems

simple. If fact, it forces the modeler to use a different modelling tech-nique. We will see how to implement such requirements in later chap-ters (see, for instance, Section 4.5). There are two valid approaches tomodel such requirements properly: Integer Programming and Con-straint Programming. The reader is encouraged to spend some timetrying to model such constraints to develop intuition into the difficul-ties. The key, and the reason that this is a beast of an entirely differentilk is that the change is not uniquely quantitative (as much as, or twicethe amount of) but is additionally qualitative: we transition betweenhaving an element and not having that element.

1.1.3 Structure of the problems under consideration

Problems with the structure of the diet problem are generally known asproduct mix problems. They can be presented in various ways but if theycan be fitted into the abstract Table 1.2 they can all be handled in the mannerdescribed in this section. Of course, it may be that some of the columns orrows are missing (no cost, or no price, or no maximum demand, etc. . . ) Thatonly simplifies the model.

The decision variable indicates the amount of product needed and theconstraints indicate availabilities of the raw material or, equivalently, ca-pacities of the processing units as well as demand bounds. The objectivesare often profits to maximize or costs to minimize or, simply, quantity toproduce.

Here are a few instances to help the reader recognize the underlyingstructure. The reader is encouraged to marshall the problems into the for-mat of Table 1.2 by inventing numbers. 4

4The web site www.practicalopt.com will generate some of these variations on de-mand.


• A factory is producing cement of various types. Each product is com-posed of the same elements, but in various quantities and we haveon hand a limited supply of each of these elements, each with a cost.To each final product is associated a profit. What is the best mix ofproduct to produce to maximize profit?

• A Florida based fruit company produces orange drinks, juices andconcentrates for various markets. The raw material for all productsare oranges, sugar, water and time, in various quantities, some posi-tive and some negative (producing orange drinks requires water; pro-ducing concentrates generates water). Given certain availabilities, howmuch can they produce to maximize profit?

• A toy manufacturer produces a number of different toys. Each is com-posed of a number of basic material and, in addition, requires specialprocessing (assembling, painting, boxing). The processing is done onspecialized machines and has a duration. Since the manufacturer haslimited supplies of material and machines, which can only operate acertain number of hours per day, how many toys can be produce?

• A fertilizer company, Bush, Rove and Cie, (BR & Co.) has two prod-ucts, a high-phosphate blend and a low-phosphate blend. These areproduced by mixing different raw materials in various quantities.

They can procure, from their own subsidiaries, at most some amount,of each raw material per day at a fixed internal cost. This cost includeslabour, power, depreciation, delivery, bribes, etc. . . In addition, themixing process incurs a certain cost per ton for each product.

Both products are sold to a wholesaler, Fox inc., at a fixed price. More-over, The wholesaler has agreed to buy all the production BR & Co.can muster. How much of each fertilizer should they produce?

• Moonbucks sells half-kilo bags of coffee in three blends, House, Spe-cial and Gourmet which sell at different prices per bag. Each of thethree blends are made up of Colombian, Cuban and Kenyan coffeebeens in various proportions. Moonbucks has on hand some Colom-bian, Cuban and Kenyan. How much of each blend should they bagto maximize revenues?

4.7. SPORTS TIMETABLING 167

TABLE 4.23: Example of Sports Timetabling Data

(Intra Inter G/W Weeks) { 2; 1; 1; 19 }Division 0 teams { 0; 1; 2; 3; 4; 5; 6 }Division 1 teams { 7; 8; 9; 10; 11; 12; 13 }

4.7 Sports TimetablingBy sports timetabling, we mean here the construction of a schedule of gamesfor a league16. If you do not care about spectator sports, read on anywayas the problem is interesting, very difficult, and leads us into the fascinat-ing and complex area of relaxation tightening that can be applied to othercomplex problems.

Here is the generic problem we will try to solve: the league has a num-ber of divisions, each with a certain number of teams. The league specifiesthe number of times, during a complete season, each team must face eachother team of the same division and of each other division as well as themaximum number of games in a week each team will play.

For a simple instance we will use to illustrate, see Table 4.23. The "Intra"parameter is the number of times each team faces each other team of thesame division. The "Inter" is for each team of other divisions. "G/W" isthe number of games per week of each team and "Weeks" is the number ofweeks of the season.

4.7.1 Constructing a model

4.7.1.1 Decision variables

What is the end result of this model? A calendar, of sorts. Something thatwill display that on Week 5, for instance, Teams 1&3, Teams 2&7 etc . . . arefacing each other. And this, for every week of the season. How can weencode this information? One possibility is for a three-dimensional binaryvariable xi,j,w where i and j are team indices (i < j) and w is a week index.The interpretation is, for example if x2,5,13 is one, then teams 2 and 5 meetduring week 13.

Does this strike the reader as profoundly inefficient? It is!It has the redeeming value that some of the constraints will be wonder-

fully simple to express. If this model works, we are done. If not, then wecan try harder. Let us pursue this avenue further.

16Think NBA if you are US American, NHL if Canadian, ARL if Australian or EPL ifyou are the colonizer of the previous three.

168 CHAPTER 4. MIXED LINEAR MODELS (MIP)

4.7.1.2 Constraints

The first constraint is that we have a fixed number (say nA) of intra-divisiongames between teams (say Td) of each division

∑w

xi,j,w = nA ∀i ∈ Td; ∀j ∈ Td; i < j; ∀d ∈ D

The inter-division constraint is similar. For number of games nR,

∑w

xi,j,w = nR ∀i ∈ Td; ∀j ∈ Te; ∀d ∈ D; ∀e ∈ D; d < e

The number of games per week that a team plays is actually an upperbound. Imagine a simple boundary case of one division with three teamsand one game per week. One of the teams cannot possibly play. Thereforewe need an inequality. For number of games nG,

∑i<j

xi,j,w + ∑j<i

xj,i,w ≤ nG ∀i ∈ T; ∀w ∈W

Notice the two sums. Since we fix the team i, we must look at the gameswith teams with both larger and smaller ordinals.

4.7.1.3 Objective function

This problem is complex enough that even feasibility is difficult. Moreoverthe possible objective functions likely vary considerably with leagues. But,just for the sake of illustration, let us assume that we want, as much aspossible, to push intra-divisional matches towards the end of the calendar.The later, the better.

Let us consider two teams of the same division, i and j. If they face eachother during week w, then the variable xi,j,w will be one. How can we put aweight on this according to ’lateness’? We could multiply by w. This leadsus to the following objective

∑w∈W

∑dinD

∑i∈Td

∑j∈Td|i<j

wxi,j,w

Unfortunately, this objective performs rather badly sometimes. For our pur-poses, all solutions that have intra-divisional games towards the end aregood. There is no reason to favour the last week over the second to last


week. So let us do some calculations and compute the number of weeks re-quired for intra-divisional games. For nA games and |Td| teams in a divisionand a maximum of nG games, we get that we need nw weeks,

nw =|Td|nA

nG

So if we assign one for intra-divisional games in the last nw weeks, and zerootherwise, we obtain the objective

|W|

∑w=|W|−nw

∑d∈D

∑i∈Td

∑j∈Td|i<j

xi,j,w

which performs much better, usually17. Notice that this computation of therequired number of weeks is not always correct; it can be off by one but itserves our purposes.

4.7.1.4 Executable model

Let us translate this into an executable model. It will receive a list of divi-sions, each containing the teams of that division. It would be simpler if alldivisions had the same number of teams, but that is never the case.

It also accepts a list of parameters, the number of intra-divisional games,nbIntra, of inter-divisional games, nbInter, of games per week for a team,nbPerWeek (note that this has to be a maximum, not a strict constraint) and,finally the number of weeks of the season, nbWeeks.

CODE 4.19: Sports Timetabling model1 def solve_model(Teams,(nbIntra,nbInter,nbPerWeek,nbWeeks)):2 nbTeams = sum([1 for sub in Teams for e in sub])3 nbDiv,Cal = len(Teams),[]4 s = newSolver(’Sports␣schedule’, True)5 x = [[[s.IntVar(0,1,’’) if i<j else None for _ in range(nbWeeks)]6 for j in range(nbTeams)] for i in range(nbTeams-1)]7 for Div in Teams:8 for i in Div:9 for j in Div:

10 if i<j:11 s.Add(sum(x[i][j][w] for w in range(nbWeeks)) == nbIntra)

17For the theoretically-minded: the primal-dual gap is smaller; optimality detection iseasier.


12 for d in range(nbDiv-1):13 for e in range(d+1,nbDiv):14 for i in Teams[d]:15 for j in Teams[e]:16 s.Add(sum(x[i][j][w] for w in range(nbWeeks)) == nbInter)17 for w in range(nbWeeks):18 for i in range(nbTeams):19 s.Add(sum(x[i][j][w] for j in range(nbTeams) if i<j) +20 sum(x[j][i][w] for j in range(nbTeams) if j<i )<=nbPerWeek)21 Value=[x[i][j][w] for Div in Teams for i in Div for j in Div \22 for w in range(nbWeeks-len(Div)*nbIntra/nbPerWeek,nbWeeks) \23 if i<j]24 s.Maximize(sum(Value))25 rc = s.Solve()26 if rc == 0:27 Cal=[[(i,j) for i in range(nbTeams-1) for j in range(i+1,nbTeams)28 if SolVal(x[i][j][w])>0] for w in range(nbWeeks)]29 return rc,ObjVal(s),Cal

Line 5 declares our decision variables. Note that this is a list of lists oflists. The first dimension is one less than the number of teams since weonly will consider matches i vs j where i < j. The second dimension is thenumber of teams but note that half of the entries (below the diagonal) willnever be used so we set them to None. The last dimension is the number ofweeks.

Line 7 starts a loop to set the number of intra-division games. We loopon each division, and then on every pair (i, j) of teams in the division, re-specting the i < j condition to only use the upper triangle.

Similarly for the loop starting at 12 where we loop on each division, thenon every division with a larger ordinal, then for every pair of teams, each inone of the two divisions.

Finally, after we solve, we massage the solution to return to the caller ameaningful result, a list of matches, indexed my the week’s ordinal. For oursmall instance, a result is shown in Table 4.24.

This approach works for smallish instances but will not scale very wellas the reader can attest by trying larger instances. (Try something the sizeof a professional league and be prepared to wait a while for a solution.)The problem stems from the interaction between the model feasible solu-tion space and the techniques used by integer programming solvers to findoptimal solutions. Solvers typically work by fixing some of the variables inthe model and then solving for the others by letting them take on fractionalsolution; iterating multiple times. For our model, this relaxation is fairly


TABLE 4.24: Optimal solution to the Sports Timetabling

Week Matches0 0 vs 12 1 vs 11 2 vs 7 3 vs 13 4 vs 9 5 vs 8 6 vs 101 0 vs 9 1 vs 10 2 vs 13 3 vs 11 4 vs 8 5 vs 12 6 vs 72 0 vs 11 1 vs 12 2 vs 8 3 vs 7 4 vs 13 5 vs 10 6 vs 93 0 vs 13 1 vs 7 2 vs 10 3 vs 9 4 vs 12 5 vs 11 6 vs 84 0 vs 8 1 vs 13 2 vs 9 3 vs 12 4 vs 10 5 vs 7 6 vs 115 0 vs 2 1 vs 4 3 vs 5 6 vs 13 7 vs 12 8 vs 10 9 vs 116 0 vs 3 1 vs 4 2 vs 6 5 vs 9 7 vs 13 8 vs 11 10 vs 127 0 vs 4 1 vs 8 2 vs 3 5 vs 6 7 vs 13 9 vs 10 11 vs 128 0 vs 1 2 vs 12 3 vs 6 4 vs 5 7 vs 8 9 vs 11 10 vs 139 0 vs 5 1 vs 6 2 vs 4 3 vs 10 7 vs 11 8 vs 12 9 vs 13

10 0 vs 6 1 vs 3 2 vs 5 4 vs 11 7 vs 10 8 vs 13 9 vs 1211 0 vs 1 2 vs 6 3 vs 4 5 vs 13 7 vs 11 8 vs 12 9 vs 1012 0 vs 2 1 vs 5 3 vs 8 4 vs 6 7 vs 9 10 vs 11 12 vs 1313 0 vs 6 1 vs 9 2 vs 3 4 vs 5 7 vs 12 8 vs 11 10 vs 1314 0 vs 5 1 vs 6 2 vs 11 3 vs 4 7 vs 10 8 vs 13 9 vs 1215 0 vs 4 1 vs 3 2 vs 5 6 vs 12 7 vs 9 8 vs 10 11 vs 1316 0 vs 7 1 vs 2 3 vs 5 4 vs 6 8 vs 9 10 vs 11 12 vs 1317 0 vs 3 1 vs 2 4 vs 7 5 vs 6 8 vs 9 10 vs 12 11 vs 1318 0 vs 10 1 vs 5 2 vs 4 3 vs 6 7 vs 8 9 vs 13 11 vs 12

weak. We will not go into the details of why, but we will see how to fix it,once we realize that the solvers is slow to solve. The key insight is that wecan easily add more constraints.

Here is an example. Imagine an instance with one game per week andconsider three teams, say i, j and k. If the decision variable is allowed totake on fractional values at some point during the execution, then, for agiven week w it could happen that :

xi,j,w =12

xi,k,w =12

xj,k,w =12

Note that this solution is allowed by the constraint that says "One game perweek per team." since

xi,j,w + xi,k,w = 1

xi,j,w + xj,k,w = 1

xi,k,w + xj,k,w = 1


TABLE 4.25: Bounds on small sums of tuples of decision variables

nb of teams nb of games per week bound on sum3 1 1

2 34 1 2

2 43 6

5 1 22 53 74 9

But we know that this is not a valid solution since the sum of the threevariables must not exceed one. If i and j face each other, then k cannot faceeither of them. Similarly for the pair (i, k) and for (j, k). Therefore, knowingthat there is only one game per week, we could add a constraint for everytriple of team i, j, k for for every week w

xi,j,w + xi,k,w + xj,k,w ≤ 1

Note that this constraint, if the variables are integers, is redundant. But itis a valid constraint for nevertheless and it is useful for fractional values,which is happening internally in the solver. We can also consider tuples offour or even five teams. And number of games per week of two or three. SeeTable 4.25 for the bounds given small numbers of teams and of games perweek. Note that many of these bounds will never be violated by fractionalsolutions so they are not very useful for our purposes.

The number of those additional constraints grows fast. The approachwill therefore add a considerable number of constraints to the model. If thatcauses solvers to slow down unacceptably, an alternative is the approach weused to solve the TSP: adding only the constraints that we need. We woulddo that by solving the relaxation, looking for tuples violating the bound,and adding those. This is the aim of Code 4.20. This is an example of howeasily one can add relaxation tightening constraints to a model written withOR-Tools.

CODE 4.20: Sports timetabling with additional cuts1 def solve_model_big(Teams,(nbIntra,nbInter,nbPerWeek,nbWeeks)):2 nbTeams,nbDiv,cuts = sum([1 for sub in Teams for e in sub]),len(

Teams),[]


3 for iter in range(2):4 s = newSolver(’Sports␣schedule’, False)5 x = [[[s.NumVar(0,1,’’) if i<j else None for _ in range(nbWeeks)]6 for j in range(nbTeams)] for i in range(nbTeams-1)]7 basic_model(s,Teams,nbTeams,nbWeeks,nbPerWeek,nbIntra,\8 nbDiv,nbInter,cuts,x)9 rc = s.Solve()

10 bounds = {(3,1):1, (4,1):2, (5,1):2, (5,3):7}11 if nbPerWeek <= 3:12 for w in range(nbWeeks):13 for i in range(nbTeams-2):14 for j in range(i+1,nbTeams-1):15 for k in range(j+1,nbTeams):16 b = bounds.get((3,nbPerWeek),1000)17 if sum([SolVal(x[p[0]][p[1]][w]) \18 for p in pairs([i,j,k],[])])>b:19 cuts.append([[i,j,k],[w,b]])20 for l in range(k+1,nbTeams):21 b = bounds.get((4,nbPerWeek),1000)22 if sum([SolVal(x[p[0]][p[1]][w]) \23 for p in pairs([i,j,k,l],[])])>b:24 cuts.append([[i,j,k,l],[w,b]])25 for m in range(l+1, nbTeams):26 b = bounds.get((5,nbPerWeek),1000)27 if sum([SolVal(x[p[0]][p[1]][w]) \28 for p in pairs([i,j,k,l,m],[])])>b:29 cuts.append([[i,j,k,l,m],[w,b]])30 else:31 break32 s = newSolver(’Sports␣schedule’, True)33 x = [[[s.IntVar(0,1,’’) if i<j else None for _ in range(nbWeeks)]34 for j in range(nbTeams)] for i in range(nbTeams-1)]35 basic_model(s,Teams,nbTeams,nbWeeks,nbPerWeek,nbIntra,\36 nbDiv,nbInter,cuts,x)37 rc,Cal = s.Solve(),[]38 if rc == 0:39 Cal=[[(i,j) for i in range(nbTeams-1) for j in range(i+1,nbTeams)40 if SolVal(x[i][j][w])>0] for w in range(nbWeeks)]41 return rc,ObjVal(s),Cal

The code starts with a loop on line 3 that will run a specific numberof times, solving the model with fractional solutions. Line 8 is essentiallyall the constraints of Code 4.19 (with the additional cuts) packaged in a


procedure because we will need to use it multiple times, with fractionalvariables within the loop and finally with integer variables after the loop.After each solve we consider tuples of teams, see if the sum of their decisionvariables exceed the prescribed bound, and add their ordinal, along withthe week under consideration and the bound to a list of cuts if it does.

Finally, we create an integer solver instance at line 32, add all the cutspreviously found and solve for real. The routine to add the cuts is simply

CODE 4.21: Cuts adding routine1 for t,w in cuts:2 s.Add(s.Sum(x[p[0]][p[1]][w[0]] for p in pairs(t,[])) <= w[1])

Where the pairs function generates all ordered pairs from ordered tuple t

CODE 4.22: Ordered pairs generation1 def pairs(tuple, accum=[]):2 if len(tuple)==0:3 return accum4 else:5 accum.extend((tuple[0],e) for e in tuple[1:])6 return pairs(tuple[1:],accum)

Let us stress that, contrary to TSP, where the subtour elimination constraintsare required for the model to be valid, the constraints we are adding hereare not required; they simply are added to nudge the solver in the right di-rection and accelerate the solution process. Therefore, for some solvers theywill help tremendously; for others they will be useless; they might actuallyslow the whole process down. Without deep knowledge of the internalworkings of a particular solver, the effect on runtime is nearly impossibleto predict18. The point is that, once a modeller is aware of this techniqueof relaxation tightening, he may easily try it on a particular combination ofmodel-solver that is proving recalcitrant.

4.7.2 Variations

The variations on this model are multiple. Some of them affect the objectivefunction (or, equivalently, are handled as soft constraints); some are hardconstraints; some could be either.

18Actually, even with deep internal knowledge, it may be near impossible to tell. Tryingthe approach to see if it works is orders of magnitude easier than reading the entrails ofcurrent integer solvers. Written in C or worse C++, they have layers upon layers of complexcut generation routines with subtle interactions, not to mention cruft accumulated overyears of development and debugging.


• There could be a list of pairs (week, team, team) with the goal of hav-ing this specified match during that specified week.

• Instead of pushing intra-divisional matches towards the end, we couldbe asked to follow a specific pattern say Intra-Intra-Inter.

• We could be asked to spread out (or, bunch in) the multiple matchesbetween pairs of teams.

• Instead of ’weeks’ we could be asked to schedule on specific dates.

• We could add the concept of "Home" and "Away" games with the un-derstanding that the number of "Home" games is fixed.

• There could also be a pattern of "Home" and "Away" games to follow.This might even be considered in the context of team cities with aneye towards a "reasonable" travel schedule. (What we are describinghere is the addition of multiple TSP layers on top of an already difficulttimetabling problem! Not for the faint of heart).


4.8 PuzzlesThere is a long tradition in Constraint Programming of solving puzzles. Be-cause it is both amusing and educational. Solving puzzles using IntegerProgramming is not tried as often, yet it can be as amusing and educationalif sometimes more difficult. Let us not be deterred by the difficulty. Thetricks used in puzzles and the mental gymnastics used to model the prob-lems can be used for "real" problems later.

4.8.1 Pseudo-chess problems

As a warm-up, let us consider a square chessboard of some specified sizen on which we wish to place as many rooks as possible so that no rook isattacking any other.19

The question to answer is "Where to place the rooks?". Therefore an an-swer must be a set of positions occupied by rooks. Since the chessboardis square, an obvious formulation of the decision variable is a two dimen-sional array of binary variables.

xi,j ∀i ∈ {0, . . . , n− 1}, ∀j ∈ {0, . . . , n− 1},

If x2,5 is one, then there is a rook at position 2, 5.The objective function is simple: since we want to place as many rooks

as possible, we sum our decision variables.

∑i

∑j

xi,j

Now what would the constraints be to prevent a rook from attackinganother? Rooks attack anything in the same column or row. So, we needto have at most one rook per column and per row. This is a constraint weknow well: the one-out-of-n constraint.

∑i

xi,j ≤ 1 ∀j ∈ {0, . . . , n− 1}

∑j

xi,j ≤ 1 ∀i ∈ {0, . . . , n− 1}

We have all we need. Let us translate this into executable code.19A rook attacks any piece in the same column or row.

4.8. PUZZLES 177

TABLE 4.26: An optimal solution to the Maxrook puzzle

1 2 3 4 5 6 7 81 R2 R3 R4 R5 R6 R7 R8 R

CODE 4.23: Maxrook model1 def get_row(x,i):2 return [x[i][j] for j in range(len(x[0]))]3 def get_column(x,i):4 return [x[j][i] for j in range(len(x[0]))]5 def solve_maxrook(n):6 s = newSolver(’Maxrook’,True)7 x = [[s.IntVar(0,1,’’) for _ in range(n)] for _ in range(n)]8 for i in range(n):9 k_out_of_n(s,1,get_row(x,i),’<=’)

10 k_out_of_n(s,1,get_column(x,i),’<=’)11 Count = s.Sum(x[i][j] for i in range(n) for j in range(n))12 s.Maximize(Count)13 rc = s.Solve()14 xx = [[[’␣’,’R’][int(SolVal(x[i][j]))] \15 for j in range(n)] for i in range(n)]16 return rc,xx

We used our k_out_of_n routine along with a couple of utility functions:one to extract all the row variables of a given row and one to extract all thecolumn variables. Running Code 4.25 on a board of size 8 can produce asolution as in 4.26. It can solve boards of size 128 with the same ease. So letus consider a slightly more difficult problem, the famous N-Queens.

This is the same problem but we are asked to place queens instead ofrooks. Queens attack on the diagonals as well as on the rows and columns.All we need is some convention on naming the diagonals and a function toextract them. To make this interesting, let us generalize our maxrook to amaxpiece accepting the type of chess piece to place.


CODE 4.24: Maxpiece helper functions1 def get_se(x,i,j,n):2 return [x[i+k % n][j+k % n] for k in range(n-i-j)]3 def get_ne(x,i,j,n):4 return [x[i-k % n][j+k % n] for k in range(i+1-j)]

CODE 4.25: Maxpiece model (Queen, rook, bishop)1 def solve_maxpiece(n,piece):2 s = newSolver(’Maxpiece’,True)3 x = [[s.IntVar(0,1,’’) for _ in range(n)] for _ in range(n)]4 for i in range(n):5 if piece in [’R’ ,’Q’]:6 k_out_of_n(s,1,get_row(x,i),’<=’)7 k_out_of_n(s,1,get_column(x,i),’<=’)8 if piece in [’B’, ’Q’]:9 for j in range(n):

10 if i in [0,n-1] or j in [0,n-1]:11 k_out_of_n(s,1,get_ne(x,i,j,n),’<=’)12 k_out_of_n(s,1,get_se(x,i,j,n),’<=’)13 Count = s.Sum(x[i][j] for i in range(n) for j in range(n))14 s.Maximize(Count)15 rc = s.Solve()16 xx = [[[’␣’,piece][int(SolVal(x[i][j]))] \17 for j in range(n)] for i in range(n) ]18 return rc,xx

We decided on naming diagonals by either SE, for southeast or NE fornortheast and created two utilities to extract the corresponding variables,get_se and get_ne. We can execute this for Queens, Rooks and Bishops. Asolution for each is shown in Table 4.27.

Note that there is an obvious generalization to any piece, since each oc-cupied position on the board describes a set of positions, hence variablesand the sum of those must be one.

Looking at the example solutions two obvious questions arise: Can weget all solutions? Can we get "interesting" solutions? We will leave the firstquestion aside for now and consider the second. What would constitute aninteresting solution? Maybe one that exhibits some symmetry. We couldtry to minimize the sum of the distances between pieces, or the maximumdistance and see what happens. But enough of pseudo-chess.

4.8. PUZZLES 179

TABLE 4.27: An optimal solution to the N-Queens and max Bishops puz-zles.

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 81 Q 1 B B B2 Q 2 B3 Q 3 B4 Q 4 B5 Q 5 B6 Q 6 B7 Q 7 B8 Q 8 B B B B B

4.8.2 Sudoku

The sudoku puzzles is as follows: given a 9 by 9 grid, partially filled withnumbers in the range 1 to 9, fill the rest of the grid such that:

• every row contains all numbers 1 to 9;

• every column contains all numbers 1 to 9;

• every 3 by 3 disjoint subgrid contains all numbers 1 to 9.

We can we represent a solution by specifying, for every grid position,which numbers is in there. So a simple decision variable is:

xi,j ∈ {1, . . . , 9} ∀i ∈ [1, 2, 3]∀j ∈ [1, 2, 3]

The constraints are interesting. Each one of them is of the form :

• Given this set of nine positions, all numbers from 1 to 9 must appear.

In constraint programming, this is handled by a single constraint, usuallynamed the all_different constraint. We will create a simplified equivalentconstraint for sudoku and improve it for the next puzzle.

For every one of our variable xi,j we will create an array vijk of length 9

of binary variables. Each of these is an indicator for the value k. So that weadd the constraint

xi,j = vij1 + 2vij

2 + 3vij3 + . . . + 8vij

8 + 9vij9 (4.23)

Then, for every set S of variables that need to be all different, we will ensurethat the sum of the corresponding indicator variables sums to one.


This is a feasibility problem so no objective function is required. So weare ready to create an executable model. We will use our previously definedget_row and get_column to which we are adding a get_subgrid.

CODE 4.26: Some helper functions for sudoku1 def get_subgrid(x,i,j):2 return [x[k][l] for k in range(i*3,i*3+3) for l in range(j*3,j*3+3)]3 def all_diff(s,x):4 for k in range(1,len(x[0])):5 s.Add(sum([e[k] for e in x]) <= 1)

CODE 4.27: Sudoku model1 def solve_sudoku(G):2 s,n,x = newSolver(’Sudoku’,True),len(G),[]3 for i in range(n):4 row=[]5 for j in range(n):6 if G[i][j] == None:7 v=[s.IntVar(1,n+1,’’)]+[s.IntVar(0,1,’’) for _ in range(n)]8 s.Add(v[0] == sum(k*v[k] for k in range(1,n+1)))9 else:

10 v=[G[i][j]]+[0 if k!=G[i][j] else 1 for k in range(1,n+1)]11 row.append(v)12 x.append(row)13 for i in range(n):14 all_diff(s,get_row(x,i))15 all_diff(s,get_column(x,i))16 for i in range(3):17 for j in range(3):18 all_diff(s,get_subgrid(x,i,j))19 rc = s.Solve()20 return rc,[[SolVal(x[i][j][0]) for j in range(n)] for i in range(n)]

The routine accepts a grid of data with either a number or None to indicatethat it must be filled.

Most of the work is to create a clean set of decision variables in the loopfrom line 3 to line 12: a three dimensional array indexed by the position onthe grid for the first two dimension. At index 0 in the third dimension isour real decision variable, holding the value that the grid will hold (eitherbecause it is data or after the solution process); each other index from 1 to9 holds the corresponding indicator variable. As we create the variables weadd the value constraint of Equation (4.23) at line 8.

4.8. PUZZLES 181

TABLE 4.28: Solution to a sudoku puzzle

1 2 5 8 3 7 6 9 44 7 6 2 1 9 8 3 59 3 8 4 6 5 7 2 18 6 3 7 4 1 9 5 22 5 1 6 9 3 4 7 87 4 9 5 8 2 1 6 35 8 4 9 2 6 3 1 76 1 2 3 7 4 5 8 93 9 7 1 5 8 2 4 6

After the variable declaration we call the all_diff function of each row,column and subgrid. This function is a simple k_out_of_n for each value inthe range 1 to 9.

Finally we return the grid values (not the eight hundred or so indicatorvariables!) As an example see Table 4.28. The data are in bold.20

4.8.3 Send more money!

Here is a crypt-arithmetic puzzle, famous in the Constraint Programmingcommunity: Replace each of the letters ’S’, ’E’, ’N’, ’D’, ’M’, ’O’, ’R’, ’Y’, witha distinct digit from 0 to 9 such that the following sum is correct

SEND + MORE = MONEY

There are two high-level constraints in this puzzle; the first is arithmetic:we need the equation to hold. We can do this by decomposing each integerinto its place-value. SEND is a four-digit number (presumably in base ten) soit really is:

S*1000 + E*100 + N*10 + D*1.

Each of MORE and MONEY is handle the same way. Then we constrain theequation to hold.

The second constraint is the requirement that all letter receive a distinctdigit. Here is another case where we could profitably use an all-differentconstraint, so let us generalize what we did for the pseudo-chess modelsso that we can invoke all-different in any model. Our trick relies on

20I have run this code on over twenty thousand puzzles. In most cases, the model runsin a small fraction of a second; occasionally it will take a few seconds.


each variable having an array of associated indicator variables, one for eachpotential integer value. So, in addition to our previously defined constraint,which we will slightly generalize, we need a routine for variable creation.This is the intent of newIntVar of Code 4.28.

CODE 4.28: all-different structure and constraint1 def newIntVar(s, lb, ub):2 l = ub-lb+13 x = [s.IntVar(lb, ub, ’’)]+[s.IntVar(0,1,’’) for _ in range(l)]4 s.Add(1 == sum( x[k] for k in range(1,l+1)))5 s.Add(x[0] == sum((lb+k-1)*x[k] for k in range(1,l+1)))6 return x7 def all_different(s,x):8 lb,ub=min(int(e[0].Lb()) for e in x),max(int(e[0].Ub()) for e in x)9 for v in range(lb,ub+1):

10 all = []11 for e in x:12 if e[0].Lb() <= v <= e[0].Ub():13 all.append(e[1 + v - int(e[0].Lb())])14 s.Add(sum(all) <= 1)15 def neq(s,x,value):16 s.Add(x[1+value-int(x[0].Lb())] == 0)

We notice that, although unstated, there is an additional assumption on’S’ and ’M’: they cannot take on value 0 if the numbers are truly four andfive digits long. So we should constrain them to be non-zero. Interestingly,the data structure we have chosen for the all_different implementationallows us to trivially create a disequality as we see in function neq of Code4.28.

Armed with this, we can now solve the puzzle. Its solution is shown at4.29.

CODE 4.29: Send More Money1 def solve_smm():2 s = newSolver(’Send␣more␣money’,True)3 ALL = [S,E,N,D,M,O,R,Y] = [newIntVar(s,0,9) for k in range(8)]4 s.Add( 1000*S[0]+100*E[0]+10*N[0]+D[0]5 + 1000*M[0]+100*O[0]+10*R[0]+E[0]6 == 10000*M[0]+1000*O[0]+100*N[0]+10*E[0]+Y[0])7 all_different(s,ALL)8 neq(s,S,0)9 neq(s,M,0)

10 rc = s.Solve()

4.8. PUZZLES 183

TABLE 4.29: Solution to the Send More Money puzzle

S E N D M O R Y9 5 6 7 1 0 8 2

11 return rc,SolVal([a[0] for a in ALL])

The solution is shown at Table 4.29. The reader can verify that the equationholds (9567 + 1085 = 10652).

4.8.4 Ladies and tigers

Raymond Smullyan in "The Lady or the Tiger" 21 presents a number of logicpuzzles. One chapter culminates in the following:

A prisoner is faced with nine doors, one of which he must open. Behindone door awaits a lady; behind the others, a tiger, if anything. One assumesthat the prisoner prefers opening the lady’s door to an empty room whichis preferable to a tiger’s den. What turns this into a logic puzzle is thaton each door is posted a logic statement (it can therefore be either true orfalse). Statements on rooms with tigers are false. The statement on theLady’s room is true.

• Door 1: The lady is in an odd-numbered room

• Door 2: This room is empty.

• Door 3: Either sign 5 is right or sign 7 is wrong.

• Door 4: Sign 1 is wrong.

• Door 5: Either sign 2 or sign 4 is right.

• Door 6: Sign 3 is wrong.

• Door 7: The lady is not in room 1.

• Door 8: This room contains a tiger and room 9 is empty.

• Door 9: This room contains a tiger and sign 6 is wrong.

Where is the lady?

21Dover recreational math; ISBN-13: 978-0486470276


To know where the lady awaits, we may need to know where are thetigers. So a reasonable decision variable, given a set R = {1, . . . , 9} of roomsand a set B = {1, 2, 3} of beasts (say 1 for empty, 2 for Lady and 3 for tiger):

ri ∈ B ∀i ∈ R

So that a Lady in room 5 would be indicated by r5 = 2 and a tiger in room4 would be r4 = 3. A statement about the lady being in an odd num-bered room is easy to accommodate if we declare our variables using ournewIntVar function. The associated array of indicator variables the perfecttool. For the sake of presentation, let us assume that for each ri variable wehave an array ri,j of indicator variables for j ∈ B.

Now for the logic part. We are given statements which can be true orfalse and their truth value influences the constraints. If we introduce a bi-nary variable for each statement, then we can use our reify logic to associatethe variable to each constraint. So let us introduce

si ∈ {0, 1} ∀i ∈ R

So that s2 = 1 will mean that the statement on door 2 is true.The executable model is Code 4.30. We will decompose it one constraint

at a time.

CODE 4.30: Lady or tiger model1 def solve_lady_or_tiger():2 s = newSolver(’Lady␣or␣tiger’, True)3 Rooms = range(1,10)4 R = [None]+[newIntVar(s,0,2) for _ in Rooms]5 S = [None]+[s.IntVar(0,1,’’) for _ in Rooms]6 i_empty,i_lady,i_tiger = 1,2,37 k_out_of_n(s,1,[R[i][i_lady] for i in Rooms])8 for i in Rooms:9 reify_force(s,[1],[R[i][i_tiger]],0,S[i],’<=’)

10 reify_raise(s,[1],[R[i][i_lady]],1,S[i],’>=’)11 reify(s,[1]*5,[R[i][i_lady] for i in range(1,10,2)],1,S[1],’>=’)12 reify(s,[1],[R[2][i_empty]],1,S[2],’>=’)13 reify(s,[1,-1],[S[5],S[7]],0,S[3],’>=’)14 reify(s,[1],[S[1]],0,S[4],’<=’)15 reify(s,[1,1],[S[2],S[4]],1,S[5],’>=’)16 reify(s,[1],[S[3]],0,S[6],’<=’)17 reify(s,[1],[R[1][i_lady]],0,S[7],’<=’)18 reify(s,[1,1],[R[8][i_tiger],R[9][i_empty]],2,S[8],’>=’)

4.8. PUZZLES 185

19 reify(s,[1,-1],[R[9][i_tiger],S[6]],1,S[9],’>=’)20 rc = s.Solve()21 return rc,[SolVal(S[i]) for i in Rooms],\22 [SolVal(R[i]) for i in Rooms]

At line 3 we define the range of integers that identify each door. Since theproblem is stated with rooms numbered from one, we will comply insteadof renumbering from zero as we usually do. In order to index from one, wecreate the decision variables on the next two lines as arrays where the firstelement contains None. We then define, at line 6, some constants to accessthe indicator variables of each room.

On line 7 we ensure that there is exactly one lady.All the other constraints involve a relation between a statement variable

S and a logical statement, hence our reify functions will prove invaluable.The first one is that if a room contains a tiger, its statement is false. Thestatement "If room i contains a tiger then statement i is false" is in the wrongdirection for us. We could write a new function but is is simpler to use thecontra-positive and state "If statement i is true, there is no tiger behind doori". This is an instance of a Boolean true enforcing a constraint, which line 9implements.

The next constraint "A room with a lady has a true statement on its door"is in the right direction for us: a satisfied constraint raising a Boolean, asimplemented at line 10.

"The lady is in an odd-numbered room." is simple. We need to sumthe i_lady indicator variables on odd doors and set them above one ifand only if statement one is true. The indices of odd rooms is obtainedby range(1,10,2) and the inequality

R[1][i_lady]+R[3][i_lady]+R[5][i_lady]+R[7][i_lady]+R[9][i_lady] >= 1

is reified to S[1] at line 11."This room is empty." is a simple reification to S[2] of

R[2][i_empty] >= 1

at line 12. The reader might wonder why not use equality instead of in-equality. The reason is that we know, having written the constraint reify,that equalities are more complex, introducing more auxiliary constraintsand or variables. If we are certain, as we are here, that an inequality issufficient, it is preferable to use one.

"Either sign 5 is right or sign 7 is wrong." is a disjunction of binary vari-ables with the minor difficulty that the second is negated. The transfor-mation to an algebraic statement is mechanical if we know how to negate


Booleans and implement disjunctions. We have seen the disjunctions often.Something like x1 ∨ . . . xn gets implemented as ∑i xi ≥ 1. The negation ofxi is handled by replacing xi by 1− xi. Therefore in our case we need

S[5] + (1-S[7]) >= 1

which simplifies to

S[5] - S[7] >= 0

reified to S[3] at line 13."Sign 1 is wrong." is S[1]==0 reified to S[4] at line 14. "Sign 3 is wrong."

is handled identically at line 16."Either sign 2 or sign 4 is right." is a simple disjunction so the usual

transformation to addition applies and is reified to S[5] at line 15."The lady is not in room 1." needs to reify to S[7] the constraint

R[1][i_lady] <= 0

as at line 17."This room contains a tiger and room 9 is empty." is interesting. The

sub-statements are

R[8][i_tiger] >= 1

and

R[9][i_empty] >= 1

The conjunction is handled by summing the right sides and the left sides tocreate

R[8][i_tiger] + R[9][i_empty] >= 2

reified to S[8] at line 18.Finally "This room contains a tiger and sign 6 is wrong." contains

R[9][i_tiger] >= 1

and

S[6] <= 0

We transform the latter into

- S[6] >= 0

4.8. PUZZLES 187

TABLE 4.30: Two solutions to the Lady or Tiger puzzle.

1 The lady is in an odd-numbered room. T Lady T2 This room is empty. T F Tiger3 Either sign 5 is right or sign 7 is wrong. T F4 Sign 1 is wrong. F F5 Either sign 2 or sign 4 is right. T F6 Sign 3 is wrong. F T7 The lady is not in room 1. F T Lady8 This room contains a tiger and room 9 is empty. F F Tiger9 This room contains a tiger and sign 6 is wrong. F F Tiger

then sum the two form the conjunction, reified to S[9] at line 19.And we are done in the sense that we can find a solution, for instance

the first solution of Table 4.30. But are there additional solutions and if so,how to find them? In this case22, it is simple since our only real goal is tofind the lady. Our first solution has the lady in room 1, so we can simplyadd a constraint preventing the lady to be in room 1, for instance

s.Add(R[1][i_lady] == 0)

We will get another solution of one exists or else the solver will indicatethat the problem is infeasible. We can proceed thus until we exhaust allsolutions if need be. The second solution of Table 4.30 is one such additionalsolution. (Interestingly this second solution is the unique solution if we adda constraint stating "Room 8 is not empty".)

22In general, for integer programs, it is very difficult to find all solutions; but it is possi-ble for many practical cases.

QUICK REFERENCE FOR OR-TOOLS MPSOLVER IN PYTHON 189

Quick reference for OR-Tools MPSolver in PythonThis is by no means a complete reference, but it is enough for all the modelsdescribed in the book. Also described are the wrapper functions used tosimplify model writing.

To use the LP and IP solvers a model must start with:

CODE 4.31: Library declaration.from linear_solver import pywraplp

A solver instance is created by:

CODE 4.32: Creation of a solver instance in OR-Tools.s = pywraplp.Solver(NAME,pywraplp.Solver.TYPE)

where s is the returned solver, NAME is any string and TYPE is one of

GLOP_LINEAR_PROGRAMMING LPCLP_LINEAR_PROGRAMMING LPGLPK_LINEAR_PROGRAMMING LPSULUM_LINEAR_PROGRAMMING LPGUROBI_LINEAR_PROGRAMMING LPCPLEX_LINEAR_PROGRAMMING LPSCIP_MIXED_INTEGER_PROGRAMMING MIPGLPK_MIXED_INTEGER_PROGRAMMING MIPCBC_MIXED_INTEGER_PROGRAMMING MIPSULUM_MIXED_INTEGER_PROGRAMMING MIPGUROBI_MIXED_INTEGER_PROGRAMMING MIPCPLEX_MIXED_INTEGER_PROGRAMMING MIPBOP_INTEGER_PROGRAMMING IP (Binary)

All the models in the book use GLOP for LP problems and CBC for allMIPs. This is how the current wrapper is set:

CODE 4.33: Creation of a solver instance via wrapper.s = newSolver(NAME,[False|True])

where False is the default and returns an LP solver instance (GLOP) andTrue returns a MIP solver (CBC).

To a solver instance we add decision variables by:

CODE 4.34: Continuous decision variable declaration via OR_Tools.var = s.NumVar(LOW,HIGH,NAME)


for continuous variables or

CODE 4.35: Integer decision variable declaration via OR_Tools.VAR = s.IntVar(LOW,HIGH,NAME)

for integer variables where VAR is the returned variable object, NAME is anystring. Names must be unique within a solver instance. Given the emptystring, an automatic unique name Will be internally generated; a preciousfeature, especially for routines that will be repeatedly reused within a solverinstance. The range is described by LOW, any number or -solver.infinity()and by HIGH, any number larger than LOW or solver.infinity(). It is a goodrule of thumb to restrict the range as much as possible.

The easiest way to create an array of decision variables is

CODE 4.36: Decision variable array declaration example.x = [s.NumVar(LOW,HIGH,’’) for _ in range(N)]

where N is the number of elements required in the array. Arrays are, ofcourse, indexed starting at zero. Similarly for high-dimensional arrays. Forexample, an M by N matrix is created by

CODE 4.37: Decision variable 2-D array declaration example.m = [[s.NumVar(LOW,HIGH,’’) for _ in range(N)] for _ in range(M)]

After variable declarations, constraints follow. The simplest constraintdeclaration is

CODE 4.38: Generic constraint declaration via OR-Tools.s.Add(REL)

where REL is (almost) any linear algebraic relation using decision variables,numbers, the arithmetic operators +,-,*,/ and the equality and inequalityrelationals. For example

CODE 4.39: Simple algebraic constraints in OR-Tools.s.Add(2 * x[12] + 30 * x[13] <= 100)s.Add(25 == x[100] - x[101])s.Add(x[99] >= x[100])

Never use the strict inequalities. For continuous variables, they make nosense and for integer variables, they can trivially be changed to the non-strict inequalities by adding one. Also remember that we can never useproducts of decision variables.

A useful helper function is the sum, invoked as

QUICK REFERENCE FOR OR-TOOLS MPSOLVER IN PYTHON 191

CODE 4.40: Sum operator in OR-Tools.s.Sum(LIST)

where LIST is any list (or tuple) of decision variables. This can be used, forinstance,

CODE 4.41: Examples of sum in OR-Tools.s.Add(s.Sum(x) <= 100)s.Add(s.Sum(m[i][j] for i in range(M) for j in range(N))<= 100)

In addition to the constraints, a model usually has an objective functionof either of the forms

CODE 4.42: Objective function declaration in OR-Toolss.Maximize(EXPR)s.Minimize(EXPR)

where EXPR is any linear algebraic expression in the decision variables.To invoke the solver on the model created:

CODE 4.43: Solver invocation in OR-Tools.rc = s.Solve()

where rc is the returned value and is zero if all goes well. It can be one of

OPTIMALFEASIBLEINFEASIBLEUNBOUNDEDABNORMALNOT_SOLVED

So, to be pedantic, one should check the return value against those definedconstants but the programmers at Google have followed the decades-oldtradition of returning zero when all goes well23.

After a solve, one accesses the optimal value and optimal solution via

CODE 4.44: Optimal value and optimal solution in OR-Toolsvalue = s.Objective().Value()varval = var.SolutionValue()

These are wrapped in helper functions

23I believe the tradition started with Dennis Ritchie and Unix as simplifying medicationagainst the Multics headache inducing complexities.


CODE 4.45: Wrapper functions for optimal value and solutions.value = ObjVal(s)xval = SolVal(x)

The returned variable xval will have the same dimensions as the parameterx.

In addition, the wrapper library provides the following higher-level con-straints

CODE 4.46: High-level constraintsl = k_out_of_n(s,k,x,rel=’==’)l = sosn(solver,k,x,rel=’<=’)delta = reify_force(s,a,x,b,delta=None,rel=’<=’,bnds=None)delta = reify_raise(s,a,x,b,delta=None,rel=’<=’,bnds=None,eps=1)delta = reify(s,a,x,b,d=None,rel=’<=’,bnds=None,eps=1)

where :

• k_out_of_n adds the necessary constraints to solver s so that exactly,at least or at most (depending on rel) k (a positive integer) variablesfrom the list x are allowed to be non-zero. It returns l an array ofbinary variables of the same length as x.

• sosn adds the necessary constraints to solver s so that exactly, at leastor at most (depending on rel) k adjacent variables from the list x areallowed to be non-zero. It returns l an array of binary variables of thesame length as x.

• reify_force adds the necessary constraints to solver s so that the re-lation ∑i aixi ≈ b is forced (for relation ≈ determined by rel) whend (a binary integer variable) is one. This last variable need not be de-clared ahead of the call to reify_force. It is returned whether createdinternally or not.

• reify_raise implements the opposite implication to force.

• reify calls both force and raise to implement an “if and only if”condition.

It also provides the helper function

CODE 4.47: High-level helper optimization function.bounds_on_box(a,x,b)

Where bounds_on_box finds the smallest and largest possible value of ∑ aixi−b on the domain of variable x.

practical mathematical models of optimization … · 2 chapter 0. introduction 0.1 what is this...

Documents