ieor e4008: computational discrete optimizationyf2414/slides.pdf · 2019-01-23 · want to travel...

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

IEOR E4008: Computational Discrete Optimization

Yuri Faenza – IEOR Department

Jan 23th, 2018

Yuri Faenza, Columbia University IEOR E4008: Computational Discrete Optimization

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Logistics

Instructor: Yuri FaenzaAssistant Professor @ IEOR from 2016Research area: Discrete Optimization

Schedule: MW, 10:10-11:25

Room: 303 Mudd

Office Hours: M, 5:30-7pm or by appointment – Mudd 334


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Website

Course website on Courseworks:

! Slides, lecture notes, links to further material,...! I’ll (try to) upload the slides of each lecture before class starts, so you can

print them if you want to.

Piazza: Online discussions on topics of the class.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Grading

30%: Assignments, roughly one every two weeks.

10%: Class participation.

60%: Project.


- I will list some projects~-

you cancome up with idea for

c project,and talk to me

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Examples of final projects

! How much would we save if all taxis were replaced by car sharing?

! How to solve discrete optimization problems with algorithms inspired bystatistical physics and genetics.

! How to summarize the content of documents using machine learning andsubmodular functions.

! How can we visit all streets in a neighborhood as quickly as possible?

! How can we build index funds using algorithms on graphs?


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Examples of assignments

In an image, each pixel is represented by a 3-dimensional vector in the RGBspace, i.e. each component represents the amount of one of the three primarycolors (red, green blue) that appears in the pixel. Each component has valuebetween 0 and 256. Adapt one of the algorithms seen in class to the followingproblem. You are given an image as a set of pixels in the RGB space, and aninteger k. Reduce the size of the picture by representing it using at most kcolors.


Image compression via clustering: an example

Original image, 302KB 2 colors, 13KB

16 colors, 44KB 32 colors, 73KB

Yuri Faenza, Columbia University IEORE4004: Optimization Models and Methods

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

What is this course about?

Discrete optimization: choose the solution of maximum profit (or minimumcost) from a discrete family.

Some discrete optimization problems you have probably seen in basic classes:

! Many problems on graphs (e.g. shortest path).

! Integer Programming.

min cxAx ≤ b

x ∈ Zn


÷:"

in .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

What is this course about?

Discrete optimization: choose the solution of maximum profit (or minimumcost) from a discrete family.

Some discrete optimization problems you have probably seen in basic classes:

! Many problems on graphs (e.g. shortest path).

! Integer Programming.

min cxAx ≤ bx ∈ Zn


f↳

integer I

Linearprogramming

program

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Why a class focused on Discrete Optimization?

Many applications:

production planning network designbiomedicine

DNA assembling machine learning

...

In general, very hard to solve ⇒ not a unique approach.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Example: online ads

! Where does the 97% of Google revenue come from1? Advertising!

Google Adwords:

! Company i chooses:! a monthly budget Bi ;! for each word w a cost c(i ,w) that the company is willing to pay to have

its ad shown when word w is searched on Google.

! When w is searched, Google chooses a set of ads to display. If the userclicks on the ad of company i , Google gets c(i ,w) from the company(unless the company has finished its budget Bi ).

1Source: http://www.wordstream.com/blog/ws/2011/07/18/most-expensive-keywords-google-adwords?dt=1


Bis 25$

we car di ,w ) -_ 70$-

- tire cci,

w ) = s $

= race cci ,w ) = 3$

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Which word produces the biggest revenue2? Which is the most expensive3?

2Source: https://searchenginewatch.com/2016/05/31/the-most-expensive-100-google-adwords-keywords-in-the-us/

3Source: http://www.wordstream.com/blog/ws/2011/07/18/most-expensive-keywords-google-adwords?dt=1


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

The Adwords problem

8

8

3

4

3

1

Who should we assign each word to?

What if arrivals are random?

Dr. Balasubramanian Sivan, Google Research.


WORDSTO Y OTA

CAR

BUDGET

RACE

G- H

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

The Adwords problem

8

8

3

4

3

1





TOYOTAWORDS f

CAR[ I

④O

2X CAR

2 x RACE

①STRATEGY I

:/GRCED-) Bugg , ,

. agg , q , µ , won ,

f)TO THE HIGHEST BIDDER

WHO STILL HAS

RACE ③7)G SOME BUDGET LEFT

[ II GM

can → TOYOTA → G

CAR → TOYOTA → 9

RACE → a- ti → I

RACE → c- n - a I-

10

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

The Adwords problem

8

8

3

4

3

1





TOYOTAWORDS I

L

CAR⑦

yy32x CAR

2x RACE

£

) BUDGET

①CAR → TOYOTA -79

RACE TSUcan → Gn → 3[ I[ gu RACE → TOYOTA → 3

RACE → g- n→

y

-

II

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

The Adwords problem

8

8

3

4

3

1

Who should we assign each word to? What if arrivals are random?



.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

The Stable Marriage problem

Applications include: Allocations of students to high schools in NYC, ofdoctors to hospitals, of kidney donors to patients, ...


A marriage is stable if

§ m,

w :

m prefers w to his partner

w a m "hav

x,

m ,W as above is

called a

"

blocking pain,

-

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




my W1 is a blockingpain

I

not a stable matching

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




NO BLOCKING PAIR

¥

STABLE MATCHING

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

A mathematical challenge

From a Procter & Gamble ad from 1962:

Imagine that Toody and Muldoon want todrive around the country and visit each ofthe 33 locations represented by dots on thecontest map, and that in doing so, theywant to travel the shortest possible route.You should plan a route for them from lo-cation to location which will result in theshortest total mileage from Chicago, Illi-nois back to Chicago, Illinois.

10.000 $ in 1962 correspond to ≈ 80.000 $ in today’s value.


TO¥-1

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

TSP and applications

Travelling Salesman Problem (TSP): find a tour of minimum length.

Today’s algorithms can solve problems with tens of thousands of cities, andthere are still open challenges.

Applications: Pick-up & delivery, Guiding industrial machines, Mappinggenomes, ...


FREE App SOLVING TSP INSTANCES : CONCORDE

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Discrete Optimization for Machine Learning

Those functions have diminishing returns and are called Submodular Functions.


i -Ip,,p4 ?' " '"sHHGReTurj

Let f ( S) = AREA COVERED BY ACTIVATING

seasons IN POSITION S

f ( sit ) + fcsntsfcs) + FCT)

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Discrete Optimization for Machine Learning

Those functions have diminishing returns and are called Submodular Functions.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

What is this course about? 1: Problems and Applications

! Minimum spanning tree, steiner minimum tree, bin packing. Networkdesign and beyond.

! TSP: approximation algorithms, heuristics, Integer Programming andbranch and bound. Extensions: vehicle routing problems.

! Matching problems, stable marriage problem. Applications to the NewYork school system and to the hospital-intern problem. Beyond stability.

! Online matching and online advertisement: adwords and secretaryproblems.

! Submodular functions and their applications in machine learning: MAPinference, document summarization, influence in social networks.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

What is this course about? 2: Algorithms

! For most of those problems, we will see one or more algorithms for theirsolutions. Why?

! Often, those algorithms contain ideas that are problem-independent and canbe reused for your favourite application.

! How “well” we want to solve the problem depends on the application.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

What is this course about? 3: Implementations

We will test our algorithm in practice using:

! Python,

! Gurobi,

! and their interaction.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Do we really need to learn all this?

! Objection 1: There are only a finitenumber of solutions to these problems.I don’t need a smart algorithm, I havea supercomputer!

! Objection 2: I’ll buy an expensivesolver and let it do the job!

Let’s prove those objections wrong!


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Number of solutions to TSP vs. very large sets

cities 10 33 100

tours 106 1037 10158

Number of cells in a human body: ≈ 1014.

Number of particles in the visible universe: ≈ 1089.


(,

n cities ⇒ # feasible solutions roughly n !

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Graphs, trees

! Graph G(V ,E)

! Degree

! Path

! Walk (open and closed)

! Cycle

! Tree

! Weighted graph

! Subgraph

! Induced Subgraph

! Multigraph


isIn

,. . in } n=s

↳ edges .lk#hk.vsl.k.inDoh

floss4 ofJ 3

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Graphs, trees

! Graph G(V ,E)

! Degree

! Path


! Cycle

! Tree

! Weighted graph

! Subgraph

! Induced Subgraph

! Multigraph


OY degrees

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Graphs, trees

! Graph G(V ,E)

! Degree

! Path


! Cycle

! Tree

! Weighted graph

! Subgraph

! Induced Subgraph

! Multigraph


o - o - o - O -

..

- o

repetition allowed ( of either nodes or

edges)

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Graphs, trees

! Graph G(V ,E)

! Degree

! Path


! Cycle

! Tree

! Weighted graph

! Subgraph

! Induced Subgraph

! Multigraph


as a path,

but repetitions allowed,first node # last node

/ "" " first nodes last node

'

-1

openwalk

closed walk

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Graphs, trees

! Graph G(V ,E)

! Degree

! Path


! Cycle

! Tree

! Weighted graph

! Subgraph

! Induced Subgraph

! Multigraph


/

%

every path is an open

walk

every cycle is a closed walk

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Graphs, trees

! Graph G(V ,E)

! Degree

! Path


! Cycle

! Tree

! Weighted graph

! Subgraph

! Induced Subgraph

! Multigraph


TREE

NOT A TREE

( 7 cycle )OI

CONNECTED GRAPH O

WITH NO CYCLEA TREE

° Oo ( NOT

CONNECTED )

Oo

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Graphs, trees

! Graph G(V ,E)

! Degree

! Path


! Cycle

! Tree

! Weighted graph

! Subgraph

! Induced Subgraph

! Multigraph


2

3

II

3

4 25!

weight can be on the

edges or on the nodes C or both)

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Graphs, trees

! Graph G(V ,E)

! Degree

! Path


! Cycle

! Tree

! Weighted graph

! Subgraph

! Induced Subgraph

! Multigraph


O

O

O

O

s a subgraph ofO

O

O O

[We Cdn choose

°

anyselection of nodesand edges ]

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Graphs, trees

! Graph G(V ,E)

! Degree

! Path


! Cycle

! Tree

! Weighted graph

! Subgraph

! Induced Subgraph

! Multigraph


O Vl

O✓ 3

✓

2004sthe subgraph of

VI

Oou's inducedby VI. Vz

, Vs VaGo oko

f wechoose a subset of

" vertices,

and take ¥ edgesbetween them ]

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Graphs, trees

! Graph G(V ,E)

! Degree

! Path


! Cycle

! Tree

! Weighted graph

! Subgraph

! Induced Subgraph

! Multigraph


the same

c) edge can

Xan:O:than once

¥.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

TSP: a formulation

The Travelling Salesman Problem (TSP):

! Given: A graph G (V ,E ) with weights w ≥ 0 on the edges.

! Find: a tour – i.e. a cycle passing through all vertices of the graph – ofminimum total weight.


cities

1 yroadsbetween cities

00O

① 5 t I tht 6 = 16

② 2 t h t 3 t s = 14

the 16 ⇒ we prefer ②

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Greedy algorithm for TSP

Input: A graph G(V ,E); w : E →R≥0.

L = [0].

For i = 0, . . . , n − 2:

! Pick j∈V \ L minimizingw(L[i ], j).

! Set L[i + 1] = j .

Output L.


±① IT MAY NOT GIVE You THG OPTIMUM

② IT HAV Bf FAL AWAY From Ttb OPTIMUM

But

IT IS FAST AND EASY To IMPLEMENT

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Pros and cons of Greedy

Pros:

! Fast

! Intuitively, seems to do “the right thing”.

Cons:

! We have no idea how far it is from the optimum.


ieor e4008: computational discrete optimizationyf2414/slides.pdf · 2019-01-23 · want to travel...

Documents