Hard Optimization Problems: Practical

Approach

DORIT RON

Tel. 08 934 2141Ziskind room #303 dorit.ron@weizmann.ac.il

Course outline

1st lecture: Introduction and motivation

INTRODUCTION

What is an optimization problem?

An optimization problem consist of:

Variables:

Energy functional to be minimized/maximized:

min / max

),...,,( 21 nxxxx

)(xE

Unconstrained minimizationFind the global minimum

An optimization problem consist of:

Variables:

Energy functional to minimized/maximized:

min / max

Possibly subject to: Equality constraints:

Inequality constraints:

),...,,( 21 nxxxx

)(xE

,0)( xfi

,0)( xfi

mi ,...,1

pi ,...,1

Constrained minimizationsubject to 0,0 yx

INTRODUCTION

What is an optimization problem? Examples

Example 1: 2D Ising spins Discrete (combinatorial) optimization

min -i,j> si sj

si = { +1 , -1}

+ + + + - - + -

+ + + + + - - -

- + + + - - - +

- - + + + + - -

- + + - - - - +

+ + + + + - + +

- - - + + + + +

- - - - - + + -

3D Ising model Each spin represents a tiny magnet The spins tend to align below a certain Tc

Ferromagnet – Iron at room temperature

magnet ------ Tc------ non-magnet

----|------------------------|---------------------------|--> T

room temp ferromagnetism melting 770oC 1538oC

At T=0 the system settles at its ground states

Example 2: 1D graph ordering Given a graph G=(V,), find a

permutation of the vertices that minimizes

E()= i j wi j | (i) -(j) |p

where i , j are in V and wi j is the edge weight between i and j (wi j =0 if ij is not in )

p=1 : Linear arrangement p=2 : Quadratic energy p= : The Bandwidth

ij

ijw

Minimum Linear Arrangement Problem

Minimum Linear Arrangement Problem

ij

1 2 3 4 5

ijw

Minimum Linear Arrangement Problem

ij

1 2 3 4 5

4)(

2)(

j

i

ijw

Minimum Linear Arrangement Problem

ij

ijw

1 2 3 4 5

ijw4)(

2)(

j

i

Minimum Linear Arrangement Problem

ij

ijw

1 2 3 4 5

ijw4)(

2)(

j

i

,

minimize over all : ( ) = ( ) ( )iji j

E w i j

Minimum Linear Arrangement Problem

ij

ijw

1 2 3 4 5

ijw4)(

2)(

j

i

,

minimize over all : ( ) = ( ) ( )iji j

E w i j

9)( E

General Linear Arrangement Problem

Variable nodes sizes

E(x)= i j wi j | xi -xj |p

xi = vi /2 + k:k)<i) vk

i j

xi xj

Other graph ordering problems

Minimize various functionals:

envelope size, cutwidth, profile of graph, etc. Traveling salesman problem – TSP

The Traveling Salesman Problem

The Traveling Salesman Problem

Other graph ordering problems

Minimize various functionals:

envelope size, cutwidth, profile of graph, etc. Traveling salesman problem – TSP Graph bisectioning Graph partitioning Graph coloring Graph drawing

Drawing Graphs

Drawing Graphs

Drawing Graphs

Example 3: 2D circuit placement Bottleneck in the microchip industry Given a hypergraph, find the discrete

placement of each module (gate) while minimizing the wirelength

The hypergraph for a microchip

Placement on a grid of pins

Placement on a grid of pins

Routing over the placement

Example 3: 2D circuit placement Bottleneck in the microchip industry Given a hypergraph, find the discrete

placement of each module (gate) while minimizing the wirelength

No overlap is allowed No overflow is allowed Critical paths must be shorter Leave white space for routing

Typical IBM chip ~270 meters on 1cm2

Place and route

Exponential growth of transistors for Intel processors

INTRODUCTION

What is an optimization problem? Examples Summary of difficulties

Difficulties:

Many variables: 106 , 107 … Many constraints: 106 , 107 … Multitude of local optima Discrete nature Conflicting objectives

Reasonable running time

INTRODUCTION

What is an optimization problem? Examples Summary of difficulties Is the global optimum really

needed / obtainable?

PEKO=PLACEMENT EXAMPLE WITH KNOWN OPTIMUM

Place the nodes – this is the solution

Create the net list locally and compactly

The optimum wire length – the sum of all the edges between the nodes, is known and can be proven to be minimal

SOLUTION QUALITY

INTRODUCTION

What is an optimization problem? Examples Summary of difficulties Is the global optimum really

needed / obtainable? What is expected of a “good approximate”

solution?

“Good approximate” solution

As optimal as possible:

high quality solution

Achievable in linear time

Scalable in the problem size

RUNTIME

Reality Check

RigorousOptimizationTheoremsLIMITED

Industrial Need for

FAST & GOOD

NP-CompleteIntractableProblems

HEURISTICS

INTRODUCTION What is an optimization problem? Examples Summary of difficulties Is the global optimum really

needed / obtainable? What is expected of a “good approximate”

solution?

Multilevel philosophy

MULTILEVEL APPROACH

PARTIAL DIFFERENTIAL EQUATIONS (Achi Brandt since the early 70’s) STATISTICAL PHYSICS CHEMISTRY IMAGE SEGMENTATION TOMOGRAPHY GRAPH OPTIMIZATION PROBLEMS

SOLUTION QUALITY

SOLUTION QUALITY

ORIGINAL PICTURE

ORIGINAL FENG SHUI (1) FENG SHUI (2) mPL

KRAFTWERK CAPO DRAGON OURS

OUR PLACEMENT

Course outline 1st lecture: Introduction and motivation 2nd – 4th : Local processing (relaxation) Quadratic minimization, Newton’s method, Steepest descent, Line search,

Lagrange multipliers, Active set approach, Linear programming

4th – 5th : Global approaches Simulated annealing, Genetic algorithms, Spectral method

6th : Classical geometric multigrid 7th : Algebraic multilevel 8th : Graph coarsening 9th – 12th : Advanced multilevel topics