module on large-scale integer programming & combinatorial ... · module on large-scale integer...

Module on Large-ScaleInteger Programming & Combinatorial

Optimization

Module on Large-ScaleInteger Programming & Combinatorial

Optimization

o Traveling salesman problemo Facility locationo Network design

o Traveling salesman problemo Facility locationo Network design

Three LecturesThree Lectures

Games/ChallengesApplications, Models, and

Solution Methods

Games/ChallengesApplications, Models, and

Solution Methods

Traveling Salesman Problem

Traveling Traveling Salesman Salesman ProblemProblem

1Thomas L. MagnantiThomas L. Magnanti

AgendaAgenda

o Originso Electronic Component Placemento TSP Modelo Turbine Vane Placemento Other Applications of TSPo Solution Methods

u Heuristic Methodsu Lagrangian relaxation (bounding methods)

o Originso Electronic Component Placemento TSP Modelo Turbine Vane Placemento Other Applications of TSPo Solution Methods

u Heuristic Methodsu Lagrangian relaxation (bounding methods)

2

Icosian game Icosian Icosian game game

3

William Rowan HamiltonWilliam Rowan Hamilton

Icosian game Icosian Icosian game game

Hamiltonian Path and the TSPHamiltonian Path and the TSPHamiltonian Path and the TSP4

1

20

19

2

3 45

6 7

89

10

131415

1617

18

11 12

Interest in TravelingSalesman Problem (TSP)

Interest in TravelingSalesman Problem (TSP)

o Arises in Many Applicationso Alluring (Captures Imagination)o Notoriously Difficult to Solveo Has Attracted Best Minds in

Math/CS/OR for 40 Years

o Arises in Many Applicationso Alluring (Captures Imagination)o Notoriously Difficult to Solveo Has Attracted Best Minds in

Math/CS/OR for 40 Years

5

The Traveling Salesman Problem and Electronics Assembly

The Traveling The Traveling Salesman Salesman Problem and Problem and Electronics Electronics AssemblyAssembly

6

SMT Placement MachineSMT Placement Machine

FeedersFeedersFeeders

Placement LocationsPlacement LocationsPlacement Locations

7

Placement ProblemPlacement Problem

8

Placement SequencePlacement Sequence

Home Position Home Home PositionPosition

9

Traveling Salesman InterpretationTraveling Salesman Interpretation

xjk =1 if placement k follows placement j

0 otherwise

xxjkjk = =11 if placement k follows if placement k follows placement jplacement j

0 otherwise0 otherwise

cjk cost of placing module k after placing module j

ccjkjk cost of placing module cost of placing module k after placing module jk after placing module j

10

Model IngredientsModel Ingredients

Minimize Σj Σk cjkxjkMinimize Minimize ΣΣj j ΣΣkk c cjkjkxxjkjk

Σkxjk = 1 for each jΣj xjk = 1 for each k

xjk > 0 for all j & k

ΣΣkkxxjkjk == 1 for each j1 for each jΣΣjj x xjkjk == 1 for each k1 for each k

xxjkjk >> 0 for all j & k0 for all j & k

subject tosubject tosubject to

Proper TSP Model?Proper TSP Model?Proper TSP Model?11

Assignment ProblemAssignment Problem

12

Subtour SolutionSubtour Solution

Assignment Model

+Subtour Breaking Constraints

Assignment ModelAssignment Model

++Subtour Breaking ConstraintsSubtour Breaking Constraints

Σj∈S Σk∈S xjk < |S| - 1 for all subsets S of nodes {2,3,…,n}

Σ Σjj∈∈SS Σ Σkk∈∈SS xxjkjk << |S| - 1 for all subsets |S| - 1 for all subsets S of nodes {2,3,…,n} S of nodes {2,3,…,n}

Proper TSP Model?Proper TSP Model?Proper TSP Model?1313

TSP ModelTSP Model

Implications for IC InsertionsImplications for IC Insertions

o Manual Designs ⇒ Long Timeu 10 hours for 70 to 100

components

o Better Solutionsu 10-25% improvements by

optimization

o Manual Designs ⇒ Long Timeu 10 hours for 70 to 100

components

o Better Solutionsu 10-25% improvements by

optimization

Other Applications SimilarOther Applications SimilarOther Applications Similar23

Feeder PlacementFeeder Placement

o Modeling?o Solution Methods?

u Heuristicu Optimization

o Modeling?o Solution Methods?

u Heuristicu Optimization

24

Other Applications of TSP?

Other Other Applications Applications of TSP?of TSP?

??

??

14

Machine SchedulingMachine SchedulingMachine Scheduling

Machine “Visits” JobsTravel Time = Set up TimeMachine “Visits” JobsMachine “Visits” JobsTravel Time = Set up TimeTravel Time = Set up Time

1515

Other Applications of TSPOther Applications of TSP

DepotDepotDepot

16

Other Applications of TSPOther Applications of TSP

1717

Vane Turbine PlacementVane Turbine Placement

Vane Turbine PlacementVane Turbine PlacementVane Turbine Placement

o Placement of Vanes in NozzleAssembly

o Ideally Area Between Two AdjacentVanes Should be Equal

o Two Parameters A(i) and B(i) for EachVane

o Area Between Vane i and i+1 Dependson A(i) + B(i+1)

o Min ∑[d-A(i) -B(i+1)]2

{d = target area}

o Placement of Vanes in NozzleAssembly

o Ideally Area Between Two AdjacentVanes Should be Equal

o Two Parameters A(i) and B(i) for EachVane

o Area Between Vane i and i+1 Dependson A(i) + B(i+1)

o Min ∑[d-A(i) -B(i+1)]2

{d = target area}18

Other ApplicationsOther Applications

o Analyzing the structure of crystalso Material handling in a warehouseo Clustering of data arrayso Cutting stock problemso Genome sequencingo Starlight interferometer satellite

positioningo DNA universal stringso Collecting coins from payphones

o Analyzing the structure of crystalso Material handling in a warehouseo Clustering of data arrayso Cutting stock problemso Genome sequencingo Starlight interferometer satellite

positioningo DNA universal stringso Collecting coins from payphones

Solving the TSPSolving the TSP

26 City Example26 City Example

2 5

3 0

3 5

4 0

4 5

5 0

7 08 09 01 0 01 1 01 2 0

Lo ng itude

L a t i t u d e

S E

P O

S F

L A

D E

T U

D A

H U N O

M E

A T

W A

P H

N Y

B O

P I

C L

D T

C NINS L

K S

O M

MN

C H

M L

2 5

3 0

3 5

4 0

4 5

5 0

7 08 09 01 0 01 1 01 2 0

Lo ng itude

L a t i t u d e

S E

P O

S F

L A

D E

T U

D A

H U N O

M E

A T

W A

P H

N Y

B O

P I

C L

D T

C NINS L

K S

O M

MN

C H

M L

2020

26 City Traveling Salesman Problem26 City Traveling Salesman Problem

LatitudeLatitude

Finding a Good SolutionFinding a Good Solution

q How?q How good is good?

uLP boundsuCombinatorial bounds

qq How?How?qq How good is good?How good is good?

uuLP boundsLP boundsuuCombinatorial boundsCombinatorial bounds

21

Solution MethodsSolution Methods

o Heuristicsu Growing solutions: nearest neighbor,

farthest neighbor, nearest insertionu Improvement procedures: 2-opt, 3-opt

o Optimization Methodsu Bounding: LP relaxation, Lagrangian

dualu Polyhedral methods (cutting planes)

o Heuristicsu Growing solutions: nearest neighbor,

farthest neighbor, nearest insertionu Improvement procedures: 2-opt, 3-opt

o Optimization Methodsu Bounding: LP relaxation, Lagrangian

dualu Polyhedral methods (cutting planes)

HeuristicsHeuristics

o Build Touru Nearest Neighboru Nearest/Farthest Insertion

o Improve Touru Swapping Edges

o Build Touru Nearest Neighboru Nearest/Farthest Insertion

o Improve Touru Swapping Edges

22

Insertion HeuristicsInsertion Heuristics

1

5 2

3

4 6

1

5 2

3

4 6

x

x

NearestInsertionNearestInsertion

FarthestInsertionFarthestInsertion

Tour ImprovementsTour Improvements

1

5 2

3

4 6

1

5 2

3

4 6

2-opt

Eliminate 2 arcs and reconnect

Choose bestalternative at each step

2-opt

Eliminate 2 arcs and reconnect

Choose bestalternative at each step

Exploiting Embedded StructureExploiting Embedded Structure

{ }

1 1

1

1

2 2

minimize

subject to 1 for all 1, 2, ,

1 for all 1, 2, ,

2

1 for all 2,3, ,

n n

ij iji j

nijj

n

iji

n n

iji j

iji S j S

ij

c x

x i n

x j n

x n

x S S n

x

= =

=

=

= =

∈ ∈

= =

= =

= −

≤ − ⊆

≥

∑ ∑∑∑

∑ ∑∑ ∑

K

K

K

0 and integerRedundant ConstraintRedundant Constraint

Minimum spanning tree on nodes 2 to nMinimum spanning tree on nodes 2 to n

Underlying StructureUnderlying Structure

1

5 2

3

4 6

Decomposed TourDecomposed Tour

Arc in and outof node 1

Arc in and outof node 1

Path (hence tree) on nodes 2 to nPath (hence tree) on nodes 2 to n

Lagrangian RelaxationLagrangian Relaxation

1 1 2 1

2 1

11

11

2 2

( , ) minimize 1

1

subject to 1

1

2

n n n nij ij i iji j i j

n nj ijj i

njj

nii

n niji j

ij

L u v c x u x

v x

x

x

x n

x

= = = =

= =

=

=

= =

= + − + −

=

=

= −

∑ ∑ ∑ ∑∑ ∑

∑∑

∑ ∑{ }1 for all 2,3, ,

0 and integeri S j S

ij

S S n

x∈ ∈

≤ − ⊆

≥

∑ ∑ K

Improving Lagrangian Lower BoundImproving Lagrangian Lower Bound

1

5 2

3

4 6

Solution x* to Lagrangian RelaxationSolution x* to Lagrangian Relaxation

Too many arcs outIncrease u5c5j+ u5 + vj more expensive

Too few arcs outDecrease u6c6j+ u6 + vj less expensive

Solution Approach (Dual Ascent)Solution Approach (Dual Ascent)

o Solve L(u,v)u Use costs cij+ vi + uj

u Select least cost arc out of and into node 1u Find minimal spanning tree on nodes 2 to n

(easy)o Let x* be optimal solution to L(u,v)o Using x* alter u and v to increase lower

bound L(u,v)o Iterate to solve Lagrangian dual

d = max u,v L(u,v)

o Solve L(u,v)u Use costs cij+ vi + uj

u Select least cost arc out of and into node 1u Find minimal spanning tree on nodes 2 to n

(easy)o Let x* be optimal solution to L(u,v)o Using x* alter u and v to increase lower

bound L(u,v)o Iterate to solve Lagrangian dual

d = max u,v L(u,v)

Dual Ascent in GeneralDual Ascent in General

* = minimize subject to

v cxAx bx X

=∈

ComplicatingConstraintsComplicatingConstraints

[ ]( ) minimize

subject to

L u cx u Ax b

x X

= + −

∈

{ }: , 0X x Dx d x= = ≥e.g.,

Lagrangian Dual

Dual Ascent in GeneralDual Ascent in General

If ( ) ( ) ( )k k k kL u cx u u Ax u b = + − ( ) ( )k kL u Ax u b∇ ≈ −

1 ( )k k kku u Ax u bθ+ = − −

2

( ) *

( )

kk

k k

cx u v

Ax u b

λθ

− =−

Dual Ascent ConvergenceDual Ascent Convergence

Theorem

If

and are bounded,

then c(xk) converges to v*.

1 20 2kε λ ε< ≤ ≤ −

( )kAx u b−

In practice, continue with fixed λk except half λk after some number (50?) of iterations if c(xk) doesn’t decrease

Solving Minimum Spanning TreeSolving Minimum Spanning Tree

o Order arcs from smallest to highestcosts

o Choose arcs in orderu If arc does not form an undirected

circuit with arcs already chosen, thenchoose arc;

u Otherwise eliminate arc fromconsideration

o Order arcs from smallest to highestcosts

o Choose arcs in orderu If arc does not form an undirected

circuit with arcs already chosen, thenchoose arc;

u Otherwise eliminate arc fromconsideration

Greedy (Kruskal Algorithm)Greedy (Kruskal Algorithm)

Amazing Facts!Amazing Facts!

o Greedy algorithm (and several variants)solves the minimum spanning problem

o The linear programming relaxation of theformulation we have given for theminimum spanning problem always has aninteger solution (the underlyingpolyhedron has integer extreme points)

o Greedy algorithm (and several variants)solves the minimum spanning problem

o The linear programming relaxation of theformulation we have given for theminimum spanning problem always has aninteger solution (the underlyingpolyhedron has integer extreme points)

Solving Network Flow ProblemsSolving Network Flow Problems

o Giden (Graphical Environment forNetwork Optimization)u http://giden.nwu.edu/download/index.ht

mlu Demonstration of

• Minimum spanning trees• Maximum flows• Minimum cost flows

(Based on Ahuja, Magnanti, Orlin’s book Network Flows)(Based on Ahuja, Magnanti, Orlin’s book Network Flows)

Traveling salesman problem through 15,112 cities in Germany

22.6 years ofcomputation on network of 110 Processors

See http://www.math.princeton.edu/tsp/

Traveling salesman problem through 15,112 cities in Germany

22.6 years ofcomputation on network of 110 Processors

See http://www.math.princeton.edu/tsp/

Today’s LessonsToday’s Lessons

o Traveling salesman problem arises innumerous applications

o Problem is a large-scale integer programo Many heuristic methods: often find good

solutionso Lagrangian dual (bounding) exploits

special problem structure (embeddedminimal spanning tree)

o MST is easy to solve

o Traveling salesman problem arises innumerous applications

o Problem is a large-scale integer programo Many heuristic methods: often find good

solutionso Lagrangian dual (bounding) exploits

special problem structure (embeddedminimal spanning tree)

o MST is easy to solve

We did NOT examine polyhedral (cutting plane) methodsWe did NOT examine polyhedral (cutting plane) methods