Approximation AlgorithmsBertrand Simon (University of Bremen)

June 13, 2019

Polynomial time approximationschemes (PTAS)

The Knapsack problem

The knapsack problem

Input: n items of integer size si and integer value via knapsack of integer size B

Goal: select a subset of items that fits in the knapsack(∑

si ≤ B) and maximizes the value (∑

vi)

Knapsack: first results

Input: n items of size si and value vi , a knapsack of size B

TheoremThe decision version of the knapsack problem is NP-hard.

Greedy algorithm:1 Sort items by decreasing vi/si (high value, small size first)2 k ← maximum j such that

∑i ≤ j si ≤ B

3 Put either the first k items, or the item indexed k + 1

Lemma (Proof: exercise)This is a (1/2) - approximation of the Knapsack problem.

Knapsack: first results

Input: n items of size si and value vi , a knapsack of size B

TheoremThe decision version of the knapsack problem is NP-hard.

Greedy algorithm:1 Sort items by decreasing vi/si (high value, small size first)2 k ← maximum j such that

∑i ≤ j si ≤ B

3 Put either the first k items, or the item indexed k + 1

Lemma (Proof: exercise)This is a (1/2) - approximation of the Knapsack problem.

Knapsack: an exact algorithm

Dynamic ProgrammingCore idea: rely on optimal solutions for several sub-problems.

T [i , v ]: minimum size to achieve value v using items 1 to i(if value v is not possible, T [i , v ] =∞)

How to compute the table T?I T [i , v ] = min {T [i − 1, v ];T [i − 1, v − vi ] + si}

The optimal value for the Knapsack problem is:I max {v ∈ [0;V ] | T [n, v ] ≤ B} with V =

∑i≤n

vi

Note: the solution set can be computed backwards from T [n, v∗]

Knapsack: an exact algorithm

Dynamic ProgrammingCore idea: rely on optimal solutions for several sub-problems.

T [i , v ]: minimum size to achieve value v using items 1 to i(if value v is not possible, T [i , v ] =∞)

How to compute the table T?I T [i , v ] = min {T [i − 1, v ];T [i − 1, v − vi ] + si}

The optimal value for the Knapsack problem is:I max {v ∈ [0;V ] | T [n, v ] ≤ B} with V =

∑i≤n

vi

Note: the solution set can be computed backwards from T [n, v∗]

Knapsack: an exact algorithm

Dynamic ProgrammingCore idea: rely on optimal solutions for several sub-problems.

T [i , v ]: minimum size to achieve value v using items 1 to i(if value v is not possible, T [i , v ] =∞)

How to compute the table T?I T [i , v ] = min {T [i − 1, v ];T [i − 1, v − vi ] + si}

The optimal value for the Knapsack problem is:I max {v ∈ [0;V ] | T [n, v ] ≤ B} with V =

∑i≤n

vi

Note: the solution set can be computed backwards from T [n, v∗]

Knapsack: an exact algorithm

Dynamic ProgrammingCore idea: rely on optimal solutions for several sub-problems.

T [i , v ]: minimum size to achieve value v using items 1 to i(if value v is not possible, T [i , v ] =∞)

How to compute the table T?I T [i , v ] = min {T [i − 1, v ];T [i − 1, v − vi ] + si}

The optimal value for the Knapsack problem is:I max {v ∈ [0;V ] | T [n, v ] ≤ B} with V =

∑i≤n

vi

Note: the solution set can be computed backwards from T [n, v∗]

Complexity remarks

Input: n items of size si and value vi , a knapsack of size B

TheoremThe Knapsack problem can be solved optimally in time O(nV ).

This is a NP-hard problem. . .Why don’t we have P = NP?

The input size is proportional to (logV ) and not to V=⇒ The Dynamic Programming algorithm is pseudo-polynomial

CorollaryThe Knapsack problem can be solved optimally in polynomial timeif V = poly(n).

Can we use this result to build a polynomial-time approximationalgorithm?

Complexity remarks

Input: n items of size si and value vi , a knapsack of size B

TheoremThe Knapsack problem can be solved optimally in time O(nV ).

This is a NP-hard problem. . .Why don’t we have P = NP?

The input size is proportional to (logV ) and not to V=⇒ The Dynamic Programming algorithm is pseudo-polynomial

CorollaryThe Knapsack problem can be solved optimally in polynomial timeif V = poly(n).

Can we use this result to build a polynomial-time approximationalgorithm?

Complexity remarks

Input: n items of size si and value vi , a knapsack of size B

TheoremThe Knapsack problem can be solved optimally in time O(nV ).

This is a NP-hard problem. . .Why don’t we have P = NP?

The input size is proportional to (logV ) and not to V=⇒ The Dynamic Programming algorithm is pseudo-polynomial

CorollaryThe Knapsack problem can be solved optimally in polynomial timeif V = poly(n).

Can we use this result to build a polynomial-time approximationalgorithm?

Polynomial Time Approximation Schemes

DefinitionA polynomial time approximation scheme (PTAS) is a family ofalgorithms Aε, such that for any ε > 0, algorithm Aε is a(1± ε)-approximation algorithm that has running time polynomialin the input size for any ε.

Note: the running time may be O(n1/ε).

DefinitionA fully polynomial time approximation scheme (FPTAS) is a PTASwith running time polynomial in both the input size and 1/ε.

Example: O(n3/ε4)

Polynomial Time Approximation Schemes

DefinitionA polynomial time approximation scheme (PTAS) is a family ofalgorithms Aε, such that for any ε > 0, algorithm Aε is a(1± ε)-approximation algorithm that has running time polynomialin the input size for any ε.

Note: the running time may be O(n1/ε).

DefinitionA fully polynomial time approximation scheme (FPTAS) is a PTASwith running time polynomial in both the input size and 1/ε.

Example: O(n3/ε4)

Towards an FPTAS for KnapsackRoadmap:

1 Given ε, modify the vi so that V = poly(n/ε)2 Solve optimally the modified instance using the DP3 Check the accuracy (i.e., approximation factor ≥ 1− ε)

Sketch:I Given a value µ, round all the vi to v ′i = bvi/µcI Solve this instance optimally⇒ Actual value of the obtained solution: SOL ≥ OPT − nµ.

Choose the right µ value: nµ ≤ εOPT ⇒ SOL ≥ (1− ε)OPTI Define M := maxi vi (≤ OPT )I Select µ = εM/nI Verify V ′ =

∑i v ′i ≤

∑i vi/µ ≤ nM/µ ≤ n2/ε = poly(n, ε)

Towards an FPTAS for KnapsackRoadmap:

1 Given ε, modify the vi so that V = poly(n/ε)2 Solve optimally the modified instance using the DP3 Check the accuracy (i.e., approximation factor ≥ 1− ε)

Sketch:I Given a value µ, round all the vi to v ′i = bvi/µcI Solve this instance optimally⇒ Actual value of the obtained solution: SOL ≥ OPT − nµ.

Choose the right µ value: nµ ≤ εOPT ⇒ SOL ≥ (1− ε)OPTI Define M := maxi vi (≤ OPT )I Select µ = εM/nI Verify V ′ =

∑i v ′i ≤

∑i vi/µ ≤ nM/µ ≤ n2/ε = poly(n, ε)

Towards an FPTAS for KnapsackRoadmap:

1 Given ε, modify the vi so that V = poly(n/ε)2 Solve optimally the modified instance using the DP3 Check the accuracy (i.e., approximation factor ≥ 1− ε)

Sketch:I Given a value µ, round all the vi to v ′i = bvi/µcI Solve this instance optimally⇒ Actual value of the obtained solution: SOL ≥ OPT − nµ.

Choose the right µ value: nµ ≤ εOPT ⇒ SOL ≥ (1− ε)OPTI Define M := maxi vi (≤ OPT )I Select µ = εM/nI Verify V ′ =

∑i v ′i ≤

∑i vi/µ ≤ nM/µ ≤ n2/ε = poly(n, ε)

FPTAS for KnapsackComplete algorithm

1 Let µ = εM/n, with M = maxi vi2 Let v ′i = bvi/µc3 Solve optimally this modified instance using the DP4 Return the set of items computed by the DP

Complexity: O(n3/ε)

Correctness proof (S = algorithm set, O = optimal set):∑i∈S

vi ≥∑i∈S

µv ′i ≥∑i∈O

µv ′i

≥(∑

i∈Ovi)− nµ ≥

(∑i∈O

vi)− εM

≥ (1− ε)OPT

A PTAS for the P ||Cmax problem

Definition of the problem P||Cmax

Input: m identical machines,n jobs, J with processing times p1, . . . , pn

Goal: assign each job j ∈ J to a machine i ∈ [m]minimizing the maximum load of any machineCmax := maxi∈[m]

∑j : j→i pj

TheoremP||Cmax is NP-hard.

PTAS idea [Hochbaum, Shmoys 87]I Use a subroutine needing a good makespan estimate

• Round job sizes to get a bounded number of distinct ones• Compute a schedule almost achieving the estimate

I Use binary search to guess the optimal makespan

The PTAS subroutine (1/2)

TheoremFor a given feasible makespan T and any ε > 0, the followingalgorithm returns a feasible schedule with makespan ≤ (1 + ε)T .

First step: rounding

Define Big jobs as Jbig = {j ∈ J | pj > εT}

Round down processing times:pj ∈ [εT (1 + ε)i , εT (1 + ε)i+1) → p′j = εT (1 + ε)i

Note: - New values are close to the originals: pj′ ≤ pj ≤ (1+ ε)pj′

- At most kε :=⌈log1+ε

1ε

⌉distinct new p′j ’s

The PTAS subroutine (2/2)Second step: big jobs (recall only kε distinct p′j in Jbig)On Jbig , run the following DP: (Bin packing problem)A[i1 . . . , ikε ]: number of machines needed to schedule ij jobs ofprocessing time εT (1 + ε)j for each j ≤ kε within makespan TA[i1 . . . , ikε ] = minx1...,xkε

(A[x1 . . . , xkε ] + A[i1 − x1 . . . , ikε − xkε ])Complexity: O(n2kε)I If the solution uses more than m machines, output: NOI Else, save an optimal allocation (actual makespan ≤ T (1+ ε))

Third step: complete the solutionAdd small jobs from J \ Jbig with list scheduling.I If the schedule has makespan > (1 + ε)T , output: NOI Otherwise, output the schedule

The PTAS subroutine (2/2)Second step: big jobs (recall only kε distinct p′j in Jbig)On Jbig , run the following DP: (Bin packing problem)A[i1 . . . , ikε ]: number of machines needed to schedule ij jobs ofprocessing time εT (1 + ε)j for each j ≤ kε within makespan TA[i1 . . . , ikε ] = minx1...,xkε

(A[x1 . . . , xkε ] + A[i1 − x1 . . . , ikε − xkε ])Complexity: O(n2kε)I If the solution uses more than m machines, output: NOI Else, save an optimal allocation (actual makespan ≤ T (1+ ε))

Third step: complete the solutionAdd small jobs from J \ Jbig with list scheduling.I If the schedule has makespan > (1 + ε)T , output: NOI Otherwise, output the schedule

A PTAS for P||Cmax

ConclusionIf we knew T = C∗max, we were done.

Binary searchWe find approximately correct C∗max by binary search.

A PTAS for P||Cmax

ConclusionIf we knew T = C∗max, we were done.

Binary searchWe find approximately correct C∗max by binary search.

A PTAS for P||Cmax

ConclusionIf we knew T = C∗max, we were done.

Binary searchWe find approximately correct C∗max by binary search.

Binary search initialized by:- LB: max{

∑pj/m,max pj} ≤ Cmax

- UB: list scheduling, so UB ≤ 2LBLoop: Set T = UB-LB

2 and run the previous algorithmIf NO, then LB := T , else UB := T

Check: If UB-LB ≤ εLB, output UB solution.This takes at most log2

1ε iterations.

Output makespan:

M ≤ (1+ε)Tlast ≤ (1+ε)(C∗max+εLB) ≤ (1+ε)2C∗max ≤ (1+3ε)C∗max

A PTAS for P||Cmax

Running timeThe running time of the algorithm is

O(# iteration of binary search)O(DP time) = O(log(1/ε)n2kε),

where kε =⌈log1+ε

1ε

⌉.

Thus, this algorithm is a PTAS.

What you should remember

Definition of a PTASI Family of algorithms: ε −→ (1 + ε)-approximationI FPTAS: also polynomial in 1/ε

Dynamic programmingI Type of algorithm which relies on the optimal solutions of

many subproblems

Famous NP-hard problemsI Knapsack: pseudo-polynomial algorithm, FPTASI P||Cmax: PTAS