Alexander Kononov Alexander Kononov Sobolev Institute of Sobolev Institute of

MathematicsMathematicsSiberian Branch of Russian Academy Siberian Branch of Russian Academy

of Scienceof Science

Novosibirsk, RussiaNovosibirsk, Russia

R u s s i aR u s s i a

NovosibirskNovosibirsk

How to design a PTASHow to design a PTAS

adapted from the novel byadapted from the novel byP. Schuurman and G. WoegingerP. Schuurman and G. Woeginger

directed bydirected byAlexander KononovAlexander Kononov

Garry Potter problemGarry Potter problem

Could you find a schedule for my new project with the minimal cost?

We can do that! Real sorcerers can do everything! And we guess the cost of the project will be 1000000 $.

Sounds great ! Wonderful ! Go ahead and determine this schedule!Tomorrow we start my new project!

We can not do that by tomorrow Real sorcerers can do everything!But finding the schedule is going to take us 23,5 years!

Tomorrow…2000000 $

But …I want …1000000 $

Then…after 23,5 years

The day after tomorrow 1500000 $

Three days from now

1333333 $

What if I call you up exactly X days from now

1000000(1+1/X)

NP-hard problemsNP-hard problems

Almost all interesting combinatorial problems are Almost all interesting combinatorial problems are NP-hard.NP-hard.

Nobody knows a polynomial time exact algorithm Nobody knows a polynomial time exact algorithm for any NP-hard problem.for any NP-hard problem.

If there exists a polynomial time exact algorithm If there exists a polynomial time exact algorithm for some NP-hard problem then there exists a for some NP-hard problem then there exists a polynomial time exact algorithm for many NP-hard polynomial time exact algorithm for many NP-hard problems.problems.

The most researchers guess the a polynomial time The most researchers guess the a polynomial time exact algorithm for NP-hard problems does not exact algorithm for NP-hard problems does not exist.exist.

We have to solve NP-hard problems approximately.We have to solve NP-hard problems approximately.

Approximation algorithmApproximation algorithm

An algorithm An algorithm AA is called is called ρρ--approximationapproximation algorithmalgorithm for problemfor problem ΠΠ, , if for all instances if for all instances II ofof ΠΠ it delivers it delivers a feasible solution with objective value A(a feasible solution with objective value A(I I ) such ) such thatthat

A(A(I I ) ≤ ) ≤ ρρOPT(OPT(I I ).).

Polynomial time approximation Polynomial time approximation schemescheme ( (PTAS)PTAS)

An approximation scheme for problemAn approximation scheme for problem ΠΠ is a family is a family

of of (1+(1+εε) ) –– approximation algorithmsapproximation algorithms AAεε for problem for problem ΠΠ

over all 0< over all 0< εε <1. <1. A polynomial time approximation schemeA polynomial time approximation scheme for problemfor problem

ΠΠ is an approximation scheme whose time complexity is an approximation scheme whose time complexity is polynomial in the input size.is polynomial in the input size.

A Fully polynomial time A Fully polynomial time approximation schemeapproximation scheme ( (FPTAS)FPTAS)

A fully polynomial time approximation A fully polynomial time approximation

schemescheme for problemfor problem ΠΠ is an approximation is an approximation scheme whose time complexity is polynomial scheme whose time complexity is polynomial in the input size and also polynomial in 1/in the input size and also polynomial in 1/εε..

RemarksRemarks

Running timeRunning time PTAS: | PTAS: | I I ||2/2/εε , | , | I I ||2/2/εε

1010 , (| , (| I I ||2/2/εε ))1/1/εε ..

FPTAS: | FPTAS: | I I ||22/εε, | , | I I ||/εε22, | , | I I ||77/εε33.. With respect to worse case approximationWith respect to worse case approximation an an

FPTAS is the strongest possible result that we FPTAS is the strongest possible result that we can derive forcan derive for an an NPNP–hard problem–hard problem..

PP22||||CCmaxmax

JJ={1,..., ={1,..., nn} – jobs} – jobs.. {{MM11, , MM22} – identical machines} – identical machines..

jj : : ppjj > 0 ( > 0 (j j =1,…, =1,…, nn)).. Each job has to be executed by one of two machinesEach job has to be executed by one of two machines.. All jobs are available at time 0 and preemption is not All jobs are available at time 0 and preemption is not

allowedallowed.. Each machine executes at most one job at timeEach machine executes at most one job at time.. The goal is to minimize the maximum job completion The goal is to minimize the maximum job completion

time.time.

How to get a PTASHow to get a PTAS

Simplification of Simplification of instance instance II.. PartitionPartition of of outputoutput space. space. Adding structure to the execution of an Adding structure to the execution of an algorithm algorithm AA..

Instance I Algorithm A Output A(I)

Simplification of instance Simplification of instance II The first idea is to turn a difficult instance into a more The first idea is to turn a difficult instance into a more

primitive instance that is easier to tackle. Then we use primitive instance that is easier to tackle. Then we use the optimal solution for the primitive instance to get a the optimal solution for the primitive instance to get a near optimal solution of the original instance.near optimal solution of the original instance.

Simplification

Solv

e

OPT #Translate back

OPTApp

I I #

Approaches of simplification Approaches of simplification

Rounding Rounding MergingMerging CuttingCutting AligningAligning

RoundingRounding

0 32

291612

43

RoundingRounding

0 32

291612

43

MergingMerging

0 32

29642

MergingMerging

0 32

29642 27

CuttingCutting

0 32

292

CuttingCutting

0 32

292

AligningAligning

0 32

29642

AligningAligning

0 32

29642

6+6+5+5 = 45.5

5+5+4+4 = 44.5

4+4+3+2+2 = 53

PP22||||CCmaxmax

JJ={1,..., ={1,..., nn} – jobs} – jobs.. {{MM11, , MM22} – identical machines} – identical machines..

jj : : ppjj > 0 ( > 0 (j j =1,…, =1,…, nn)).. Each job has to be executed by one of two machinesEach job has to be executed by one of two machines.. All jobs are available at time 0 and preemption is not All jobs are available at time 0 and preemption is not

allowedallowed.. Each machine executes at most one job at timeEach machine executes at most one job at time.. The goal is to minimize the maximum job completion The goal is to minimize the maximum job completion

time.time.

Lower boundLower bound

n

jjsum pp

1j

nj pp 1max max

max

*max ,

2max p

pLC sum

How to simplify an instanceHow to simplify an instance ( (II II##))

Big Big = { = { jj J J | | ppj j ≥ ≥ εεLL}} New instanceNew instance II## contains all the big jobs fromcontains all the big jobs from I.I.

Small Small = { = { j j J J | | ppj j < < εεLL}}

LetLet XX= = ΣΣjjSmall Small ppjj .. New instanceNew instance II# # containscontains XX//εεLL jobs jobs of lengthof length εεLL..

The small jobs in The small jobs in I I are first glued together to give a are first glued together to give a long job of length long job of length XX, and then this long job is cut into , and then this long job is cut into lots of chunks of length lots of chunks of length εεL L ..

I I and and II##

The optimal makespan of The optimal makespan of II# # is fairly close to is fairly close to the optimal makespan of the optimal makespan of II: :

OPT(OPT(II##) ) (1+ (1+ εε)OPT()OPT(I I ).).

ProofProof

XXii – the total size of all small jobs on machine– the total size of all small jobs on machine MMii in in

optimal schedule foroptimal schedule for II. . On On MMii, leave every big job where it is in optimal , leave every big job where it is in optimal

schedule.schedule. Replace the small jobs onReplace the small jobs on MMii byby XXii //εεLL chunks of chunks of

lengthlength εεLL.. XX11 //εεLL + + XX22 //εεLL XX11 //εεLL + + XX22 //εεLL = = XX//εεLL XXii //εεLLεεL – XL – Xi i ((XXii //εεL + L + 1) 1) εεL – XL – Xi i εεLL OPT(OPT(II##) ) OPT + OPT + εεL L (1+ (1+ εε)OPT()OPT(II))

How to solve the simplified instanceHow to solve the simplified instance

How many jobs in instanceHow many jobs in instance II##?? ppj j ≥ ≥ εεLL for all jobs infor all jobs in II##.. The total length of all jobsThe total length of all jobs in in II## :: ppsum sum 2 2LL.. The number of jobs inThe number of jobs in II## 22L/L/εεL= L= 22//εε.. The number of jobs inThe number of jobs in II## is independent ofis independent of n.n. We may simply try all possible schedulesWe may simply try all possible schedules.. The number of all possible schedulesThe number of all possible schedules 2 222//εε !! Running time is Running time is OO((2222//εε nn)!)!

How to translate solution backHow to translate solution back

LetLet σσ## be an optimal schedule for instance be an optimal schedule for instance II##.. LetLet L Lii

## be the load of machinebe the load of machine MMii inin σσ##. .

LetLet B Bii##

be the total length of the big jobsbe the total length of the big jobs on on MMii inin σσ##. .

Let XLet Xi i be the total size of the small jobsbe the total size of the small jobs on on MMii inin σσ##. .

LLii## = = BBii

## + + XXii##..

LXL

XLXX

#

2#

1

σσ##((II##)) σσ((II))

Every big job is put onto the same machine as in Every big job is put onto the same machine as in scheduleschedule σσ##..

Reserve an intervalReserve an interval of length of length XX11## + 2+ 2εεLL on machineon machine

MM11 and an interval of lengthand an interval of length XX22## on machineon machine MM22..

Pack small jobs into the reserved interval on Pack small jobs into the reserved interval on machine machine MM11 until meet some small job that does until meet some small job that does

not fit in anymorenot fit in anymore.. Pack remaining unpacked jobs into the reserved Pack remaining unpacked jobs into the reserved

interval on machine interval on machine MM22..

PTASPTAS

max,

2max p

pLOPT sum

OPTOPTOPT

LLLXBL iiii

3121

22 ###

# # 1iL OPT OPT

Structuring the outputStructuring the output

The main idea is to cut output space (i.e. the set of feasible The main idea is to cut output space (i.e. the set of feasible solutions) into lots of smaller regions over which the solutions) into lots of smaller regions over which the optimization problem is easy to approximate. Solve the optimization problem is easy to approximate. Solve the problem separately for each smaller region and taking the problem separately for each smaller region and taking the best approximate solution over all region will then yield a best approximate solution over all region will then yield a globally good approximate solution.globally good approximate solution.

1.1. PartitionPartition. .

2.2. Find representativesFind representatives..

3.3. Take the bestTake the best..

PartitionPartition

*

* - the global optimal solution

Find representativesFind representatives

**

*

*

*

**

**

* **

**

*

**

*

**

*

*

* - the global optimal solution

* - an optimal solution in his district * - a representative in his district

Take the bestTake the best

**

*

*

*

**

**

* **

**

*

**

*

**

*

*

* - the global optimal solution

* - an optimal solution in his district * - a representative in his district

PP22||||CCmaxmax

JJ={1,..., ={1,..., nn} – jobs} – jobs.. {{MM11, , MM22} – identical machines} – identical machines..

jj : : ppjj > 0 ( > 0 (j j =1,…, =1,…, nn)).. Each job has to be executed by one of two machinesEach job has to be executed by one of two machines.. All jobs are available at time 0 and preemption is not All jobs are available at time 0 and preemption is not

allowedallowed.. Each machine executes at most one job at timeEach machine executes at most one job at time.. The goal is to minimize the maximum job completion The goal is to minimize the maximum job completion

time.time.

How to define the districtsHow to define the districts

Big Big = { = { jj JJ| | ppj j ≥ ≥ εεLL}} Small Small = { = { j j JJ| | ppj j < < εεLL}} Let Let ΦΦ be the set of feasible solutions for be the set of feasible solutions for II.. Every feasible solutionEvery feasible solution σσΦΦ specifies an assignment specifies an assignment

of the of the n n jobs to the two machinesjobs to the two machines.. Define the districtsDefine the districts ΦΦ(1)(1), , ΦΦ(2)(2),…,…according to theaccording to the

assignmentassignment of big jobs to the two machinesof big jobs to the two machines:: Two Two feasible solutionsfeasible solutions σσ11 и и σσ2 2 lielie in the same district if and in the same district if and only if only if σσ11 assigns everyassigns every big job to the same machine big job to the same machine asas σσ22 does. does.

Number of districtsNumber of districts

The number of big jobs The number of big jobs 22L/L/εεL L == 22//εε.. The number of different ways for assigning The number of different ways for assigning

these jobs to two machines these jobs to two machines 2 222//εε.. The number of districts The number of districts 2 222//εε!! The number of districts depends onThe number of districts depends on εε and is and is

independent of the input sizeindependent of the input size! !

How to find good representativesHow to find good representatives

The assignments of big jobs to their machines are fixed inThe assignments of big jobs to their machines are fixed in ΦΦ((ll)).. Let OPTLet OPT((ll)) be the makespan of the best schedule in be the makespan of the best schedule in ΦΦ((ll)).. LetLet B Bii

((ll)) be the total length of big jobs assigned to machinebe the total length of big jobs assigned to machine MMii..

T T := max{:= max{BBii((11)), , BBii

((22))} } OPT OPT((ll))

The initial workload of machine The initial workload of machine MMi i isis BBii((ll))..

We assign the small jobs one by one to the machines;We assign the small jobs one by one to the machines; every every time a job is assigned to the machine with the currently time a job is assigned to the machine with the currently smaller workload.smaller workload.

The resulting scheduleThe resulting schedule σσ((ll) ) with makespanwith makespan AA((ll)) is our is our representative for the districtrepresentative for the district ΦΦ((ll))..

How close is How close is AA((ll)) to OPTto OPT((ll))

1.1. IfIf AA((ll)) = =TT, , thenthen AA((ll)) = = OPT OPT((ll))..2.2. LetLet AA((ll)) >>TT..

1.1. Consider the machine with higher workload in the Consider the machine with higher workload in the scheduleschedule σσ((ll))..

2.2. Then the last job that was assigned to the machine Then the last job that was assigned to the machine is a small job and it has length at mostis a small job and it has length at most εεLL. .

3.3. At the moment when this small job was assignedAt the moment when this small job was assigned to the machine the workload of this machine was to the machine the workload of this machine was at mostat most ppsum sum // 2. 2.

4.4. AA((ll)) ( (ppsum sum // 2) + 2) + εεL L (1 + (1 + εε)OPT )OPT (1 + (1 + εε)OPT)OPT((ll))

Structuring the execution of an Structuring the execution of an algorithmalgorithm

The main idea is to take an exact but slow The main idea is to take an exact but slow algorithm algorithm AA, and to interact with it while it is , and to interact with it while it is workingworking..

If the algorithm accumulates a lot of auxiliary If the algorithm accumulates a lot of auxiliary data during its execution, then we may remove data during its execution, then we may remove part of this data and clean up the algorithm’s part of this data and clean up the algorithm’s memory.memory.

As a result the algorithm becomes fasterAs a result the algorithm becomes faster..

PP22||||CCmaxmax

JJ={1,..., ={1,..., nn} – jobs} – jobs.. {{MM11, , MM22} – identical machines} – identical machines..

jj : : ppjj > 0 ( > 0 (j j =1,…, =1,…, nn)).. Each job has to be executed by one of two machinesEach job has to be executed by one of two machines.. All jobs are available at time 0 and preemption is not All jobs are available at time 0 and preemption is not

allowedallowed.. Each machine executes at most one job at timeEach machine executes at most one job at time.. The goal is to minimize the maximum job completion The goal is to minimize the maximum job completion

time.time.

Code of feasible solutionCode of feasible solution

LetLet σσkk be a feasible schedule ofbe a feasible schedule of k k first first

jobsjobs {1,..., {1,..., kk}. }. We encodeWe encode a feasible schedule a feasible schedule σσkk with with

machine loads machine loads LL1 1 andand LL22 by the two by the two

dimensional vectordimensional vector [[LL11, , LL22].].

LetLet VVkk be be the vector setthe vector set correspondingcorresponding to to

feasible schedulesfeasible schedules of of k k jobsjobs {1,..., {1,..., kk}. }.

Dynamic programming Dynamic programming

Input Input (( JJ={1,..., ={1,..., nn}},, pp: : JJ → → ZZ++))1)1) SetSet VV00={[0,0]}, ={[0,0]}, ii=0.=0. 2)2) While While ii n n do:do:

for every vectorfor every vector [[xx,,yy]] VVii putput [[xx + + ppii ,,yy] and] and [[xx,,y y + + ppii ]] inin VVii+1+1; ;

ii:= := i i +1;+1;3)3) Find the vectorFind the vector [[x*x*,,y*y*]] VVnn thatthat minimize the valueminimize the value max max [[xx,,yy]]VVnn

{{xx,,yy}.}.OutputOutput ([([x*x*,,y*y*])])

Running timeRunning time

The coordinates of all vectors are integer in the range The coordinates of all vectors are integer in the range fromfrom 0 0 toto ppsumsum..

The cardinality of every vector setThe cardinality of every vector set VVii is bounded from is bounded from above byabove by ( (ppsumsum))22..

The total number of vectors determined by the The total number of vectors determined by the algorithm is at mostalgorithm is at most nn((ppsumsum))22..

The running time of the algorithm is The running time of the algorithm is OO((nn((ppsumsum))22).). The sizeThe size ||II| of the input | of the input II satisfies satisfies

| |II| ≥ log(| ≥ log(ppsumsum)) = const · ln( = const · ln(ppsumsum)).. The running time of the algorithm is not polynomial The running time of the algorithm is not polynomial

of the size of the input!of the size of the input!

How to simplify the vector setsHow to simplify the vector sets

psum

psum

1

(psum, psum)

10

Δ

Δ

Δ2

Δ2

Δ3

Δ3

ΔK

ΔΔ = 1+ ( = 1+ (εε/2/2nn))

K K = = loglogΔΔ((ppsumsum)) = =

= = ln(ln(ppsumsum))/ln /ln ΔΔ ≤ ≤

≤ ≤ ((1+2((1+2n n ))//εε) ln() ln(ppsumsum))

ΔK

Trimmed vector setTrimmed vector set

psum

psum

1

(psum, psum)

10

Δ

Δ

Δ2

Δ2

Δ3

Δ3

ΔK ΔΔ = 1+ ( = 1+ (εε/2/2nn))

K K = = loglogΔΔ((ppsumsum)) = =

= = ln(ln(ppsumsum))/ln /ln ΔΔ ≤ ≤

≤ ≤ ((1+2((1+2n n ))//εε) ln() ln(ppsumsum))

ΔK

Algorithm FPTAS Algorithm FPTAS

Input Input (( JJ={1,..., ={1,..., nn}},, pp: : JJ → → ZZ++))1.1. SetSet VV00

##={[0,0]}, ={[0,0]}, ii=0.=0.

2.2. While While ii n n do:do:1.1. for every vector [for every vector [xx,,yy]] VVii

## put put

[[xx + + ppii ,,yy] and] and [[xx,,y y + + ppii ]] inin VVii+1+1; ;

2.2. ii:= := i i +1;+1;3.3. TransformTransform VVii intointo VVii

##..• Find the vectorFind the vector [[x*x*,,y*y*]]VVnn

##, , thatthat

minimize the value max minimize the value max [[xx,,yy]]VVnn##{{xx,,yy}.}.

OutputOutput ([([x*x*,,y*y*])])

Running time of FPTASRunning time of FPTAS

The trimmed vector setThe trimmed vector set VVii## contains at most contains at most

one vector in each boxone vector in each box.. There are There are KK2 2 boxes.boxes. Running time of FPTAS Running time of FPTAS OO((nKnK22).). nKnK22 = = nn((1+2((1+2n n ))//εε) ln() ln(ppsumsum))22

. . Algorithm FPTASAlgorithm FPTAS has a time complexity that has a time complexity that

is polynomial in the input size and in is polynomial in the input size and in 11//εε..

VVii andand VVii##

For every vectorFor every vector [[xx,,yy]] VVii there exists a vector there exists a vector

[[xx##,,yy##]] VVii## , , such thatsuch that xx# # ΔΔiix x andand yy# # ΔΔiiy. y.

The worst case behavior FPTASThe worst case behavior FPTAS

# #max , max , max ,n n n nx y x y x y OPT

10 211

zz

nz

n

OPTOPTn

OPTn

n

1

21

Final remarksFinal remarks

Do we consider all approachesDo we consider all approaches?? No we don’t, of courseNo we don’t, of course!!

Approximation Algorithms for NP-hard Approximation Algorithms for NP-hard problemsproblems, edited by , edited by D.HochbaumD.Hochbaum, PWS , PWS Publishing Company, 1997.Publishing Company, 1997.

V. VaziraniV. Vazirani Approximation AlgorithmsApproximation Algorithms, , Springer-Verlag, Berlin, 2001.Springer-Verlag, Berlin, 2001.

P. Schuurman, G. WoegingerP. Schuurman, G. Woeginger, , Approximation Schemes – A TutorialApproximation Schemes – A Tutorial, chapter of , chapter of the book “Lecture on Scheduling”, to appear in the book “Lecture on Scheduling”, to appear in 2008.2008.