cs483 analysis of algorithms lecture 11 – np …jmlien/teaching/09_spring_cs483/...analysis of...

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 1

CS483 Analysis of AlgorithmsLecture 11 – NP-completeness∗

Jyh-Ming Lien

April 23, 2009

∗this lecture note is based onAlgorithmsby S. Dasgupta, C.H. Papadimitriou, and U.V. Vazirani andIntro-duction to the Design and Analysis of Algorithmsby Anany Levitin.

Hard Problems

⊲ Hard Problems

Hard Problems

Search Problems

Complexity Class

Reductions

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 2

� Search problems (formal definition later):

– Search for shortest path in a graph– Search for values ofx, y andz to satisfy:x2y

12 z − xz2 = 7

– Search for a path in 3D space among obstacles– ...

� Success on solving these hard problems in polynomial time

– Greedyproperties: without looking backward or forward, MST,shortest paths

– Optimality fromsubproblems: divide and conquer, dynamicprogramming

– Convexity: gradient decent/hill climbing, linear programming

� Success on solving these hard problems is based on somespecialproperties of the problems

Hard Problems

Hard Problems

⊲ Hard Problems

Search Problems

Complexity Class

Reductions

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 3

� Failures

– Many problems require2n, n! or evennn (intractable)– For some (clearly formalized) problems we don’t even have

algorithms to solve them– (not to mention those problems that we can not even formalize)

� In this lecture, we will look at

– What are theseproblems? (Search Problems)– Terminology to classifyproblems: P vs. NP– How do you know if aproblemcan be solved efficiently?

(Problem Reduction)– Do we have any hope of solving theseproblemsefficiently?

Search Problems

Hard Problems

Hard Problems

⊲ Search ProblemsWhat Are SearchProblems?

Optimization Problems

Search Problems

Search Problems

Search Problems

Search Problems

Search Problems

Search Problems

Complexity Class

Reductions

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 4

What Are Search Problems?

Hard Problems

Hard Problems

Search Problems

⊲What Are SearchProblems?

Optimization Problems

Search Problems

Search Problems

Search Problems

Search Problems

Search Problems

Search Problems

Complexity Class

Reductions

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 5

� A Search problem has

– An instance of problemI that is input data specifying the problem– Asked to find a solutionS that meets a particular specification– Polynomial-time Checkable: There most be an algorithmC that

takesI andS and checks for correctnessefficiently, i.e., inpolynomial time

� Example: Satisfability problem

– I : (x ∨ y ∨ z)(x ∨ y)– S : {x = T, y = F, z = T}– C : (T ∨ F ∨ T )(T ∨ F )

� Example: Traveling salesman problem

– I : G = {V, E}– S : {vi, vk, · · · , vi}– C :

Optimization Problems

Hard Problems

Hard Problems

Search ProblemsWhat Are SearchProblems?

⊲ Optimization Problems

Search Problems

Search Problems

Search Problems

Search Problems

Search Problems

Search Problems

Complexity Class

Reductions

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 6

� We convert an optimization problem to a search problem

– by introduce abudgeb

– Budget does not make the problem harder or easier.

� Example: Traveling salesman problem with budgetb

– I : G = {V, E}, b– S : {vi, vk, · · · , vi}– C :

� Why do we convert an optimization problem to a search problem?

Search Problems

Hard Problems

Hard Problems

Search ProblemsWhat Are SearchProblems?

Optimization Problems

⊲ Search Problems

Search Problems

Search Problems

Search Problems

Search Problems

Search Problems

Complexity Class

Reductions

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 7

� Many problems we studied in the previous chapters are searchproblems, e.g., all-pairs shortest paths problem, single-source shortestpaths problem, minimum spanning tree, maximum flow minimum cut,matching, ...

– But why are these problem tractable? These problems seem to haveVeryLargesearch spaces

– Many algorithms seem to defeat the curse of expoentiality!

� Now, after we have seen the most brilliant successes, it’s about timefor us to face some failure in this quest.

– Satisfability (SAT, 2SAT, 3SAT)– MST and TSP (traveling salesman problem) with or without budgetb– Euler and Rudrata– Minimum cuts and balanced cuts– Integer linear programming or ILP (Ax ≤ b) and Zero-one Equations or

ZOE (Ax = 1)– Three dimensional matching– Independent set, vertex cover, clique problem (with budget b)– Longest path

Search Problems

Hard Problems

Hard Problems

Search ProblemsWhat Are SearchProblems?

Optimization Problems

Search Problems

⊲ Search Problems

Search Problems

Search Problems

Search Problems

Search Problems

Complexity Class

Reductions

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 8

� Satisfability problems (Horn-SAT, 2SAT, 3SAT, KSAT)� Horn’s formula

– Implications:(z ∧ w) ⇒ u

– Pure negative clauses:(u ∨ v ∨ y)– Horn-SAT: Solvable using greedy algorithm

� 2SAT

– in conjunctive normal form (CNF)– Each clause has two literals– example:(x ∨ y) ∧ (x ∨ z) ∧ (x ∨ w)– Can be solve in polynomail time

� 3SAT

– in conjunctive normal form (CNF)– Each clause has 3 literals– example:(x ∨ y ∨ w) ∧ (x ∨ z ∨ y) ∧ (x ∨ u ∨ w)– No polynomial time algorithm

Search Problems

Hard Problems

Hard Problems

Search ProblemsWhat Are SearchProblems?

Optimization Problems

Search Problems

Search Problems

⊲ Search Problems

Search Problems

Search Problems

Search Problems

Complexity Class

Reductions

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 9

� Euler’s tour and Rudrata’s problem

Euler’s tourvisit all edges without repeatingSolvable in polynomial time

Rudrata’s problemvisit all vertices without repeatingNo polynomial time algorithm

Search Problems

Hard Problems

Hard Problems

Search ProblemsWhat Are SearchProblems?

Optimization Problems

Search Problems

Search Problems

Search Problems

⊲ Search Problems

Search Problems

Search Problems

Complexity Class

Reductions

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 10

� Longest path (Taxicab rip-off prob-lem)

– Give a graph and two nodess andt, find a path with length at leastb

from s to t without repeating ver-tices.

– no polynomial time algorithm

� Minimum cuts and balanced cuts

– Find cuts that split the graph into twosetsS andT

– Minimum cuts problem can be solvedusing linear programming

– Balanced cuts:|S| ≥ n/3 |T | ≥n/3 and there are at mostb edges be-tweenS andT

– Balanced cuts problem has no poly-nomial time algorithm

Search Problems

Hard Problems

Hard Problems

Search ProblemsWhat Are SearchProblems?

Optimization Problems

Search Problems

Search Problems

Search Problems

Search Problems

⊲ Search Problems

Search Problems

Complexity Class

Reductions

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 11

� Three dimensional matching

� Independent set, vertex cover, clique problem (with budgetb)

– Independent set: A set ofb vertices that are not adjacent to eachother

– vertex cover: A set ofb vertices that are incident to all edges– clique: A set ofb vertices that have all possible connections

Search Problems

Hard Problems

Hard Problems

Search ProblemsWhat Are SearchProblems?

Optimization Problems

Search Problems

Search Problems

Search Problems

Search Problems

Search Problems

⊲ Search Problems

Complexity Class

Reductions

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 12

� Knapsack problem and Subset sum

– In subset sum, each item has same value and weight– Both problems have no polynomial time algorithms

� Integer linear programming or ILP (Ax ≤ b) and Zero-one Equationsor ZOE (Ax = 1)

– Simplex method is not polynomial but LP can be solved inpolynomial time

– ILP requires the values of all variables to be integer– ZOE is a special type of ILP where all values inA are 0 or 1– Both problems have no polynomial time algorithms

Complexity Class

Hard Problems

Hard Problems

Search Problems

⊲ Complexity Class

Problem Complexity

P vs. NPP vs. NP vs. NP hard vs.NP completeP vs. NP vs. NP hard vs.NP complete

Reductions

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 13

Problem Complexity

Hard Problems

Hard Problems

Search Problems

Complexity Class

⊲ Problem Complexity

P vs. NPP vs. NP vs. NP hard vs.NP completeP vs. NP vs. NP hard vs.NP complete

Reductions

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 14

� Tractable: a problem is tractable if there is an algorithm can solve theproblem deterministically inpolynomial time

� Is a problem tractable or intractable?

– yes (given an algorithm to support this answer)– no

⊲ because it’s been proved that no algorithm exists at all (e.g.,Turing’sHalting Problem.)

⊲ because it’s been be proved that any algorithm takesexponential time (Traveling Sales Men Problem)

– we have no idea...

P vs. NP

Hard Problems

Hard Problems

Search Problems

Complexity Class

Problem Complexity

⊲ P vs. NPP vs. NP vs. NP hard vs.NP completeP vs. NP vs. NP hard vs.NP complete

Reductions

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 15

� Hard problems, easy problemsHard problemsNP-complete Easy problemsP

SAT, 3SAT 2SATTraveling Salesman Problem, Rudrata pathChinese Postman Problem, Euler path

3D matching Bipartite matchingIndependent set Independent set on trees

Integer linear programming Linear programmingBalance cut Minimum cut

� P: polynomial

– Given an instanceI, we can find a polynomial time algorithm to find an solutionS

� NP: nondeterministic polynomial

– Given an instanceI and a proposed solutionS, we can find a polynomial timealgorithmC to check ifS is an solution ofI

– Remember: A problem inNP doesNOT mean it is a hard problem

� NP hard: all problems in NP can bereducedto a NP hard problem

– A NP hard problem is at least as hard as the hardest problem inNP

� NP complete: in NP and also in NP hard

P vs. NP vs. NP hard vs. NP complete

Hard Problems

Hard Problems

Search Problems

Complexity Class

Problem Complexity

P vs. NP

⊲P vs. NP vs. NP hardvs. NP complete

P vs. NP vs. NP hard vs.NP complete

Reductions

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 16

� Their relationship

P vs. NP vs. NP hard vs. NP complete

Hard Problems

Hard Problems

Search Problems

Complexity Class

Problem Complexity

P vs. NPP vs. NP vs. NP hard vs.NP complete

⊲P vs. NP vs. NP hardvs. NP complete

Reductions

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 17

� Problems in Psorting, MST, ...

� Problems in NP completeSAT, TSP, ILP, ...

� Problems in NP but not in P or in NP completefactoring, graph isomorphism

� Problems not in NPHalting problem, counting the number of perfect matching, matrix permanent,...

� P=NP? (Most Computer Scientists believe P6= NP)� There are many more complexity classes than these three (PSpace, Co-NP,

ExpTime, ExpSpace,...)In fact, there are 462 complexity classes according to the “Complexity zoo”http://qwiki.caltech.edu/wiki/ComplexityZoo (maintained by Scott Aaronson and GregKuperberg)

Reductions

Hard Problems

Hard Problems

Search Problems

Complexity Class

⊲ Reductions

Reductions

Reductions

TSP→ TSP with budgetb

Rudrata(s, t)-Path→Rudrata cycle

3SAT→ Independent SetIndependent Set→ VertexCover

Independent Set→ Clique

SAT → 3SAT

3SAT→ 3D Match

3D Match→ ZOE

ZOE→ Rudrata

Rudrata→ TSPAll Problems in NP→SAT

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 18

Reductions

Hard Problems

Hard Problems

Search Problems

Complexity Class

Reductions

⊲ Reductions

Reductions

TSP→ TSP with budgetb

Rudrata(s, t)-Path→Rudrata cycle

3SAT→ Independent SetIndependent Set→ VertexCover

Independent Set→ Clique

SAT → 3SAT

3SAT→ 3D Match

3D Match→ ZOE

ZOE→ Rudrata

Rudrata→ TSPAll Problems in NP→SAT

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 19

� If we reduce a problemA to a problemB in polynomial time (denoted asA → B, we can say thatB is as hard asA if not harder

– If A → B andA is NP-complete then we know thatB is alsoNP-complete

Reductions

Hard Problems

Hard Problems

Search Problems

Complexity Class

Reductions

Reductions

⊲ Reductions

TSP→ TSP with budgetb

Rudrata(s, t)-Path→Rudrata cycle

3SAT→ Independent SetIndependent Set→ VertexCover

Independent Set→ Clique

SAT → 3SAT

3SAT→ 3D Match

3D Match→ ZOE

ZOE→ Rudrata

Rudrata→ TSPAll Problems in NP→SAT

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 20

� Reductions between NP-complete problems

TSP→ TSP with budget b

Hard Problems

Hard Problems

Search Problems

Complexity Class

Reductions

Reductions

Reductions

⊲TSP→ TSP withbudgetb

Rudrata(s, t)-Path→Rudrata cycle

3SAT→ Independent SetIndependent Set→ VertexCover

Independent Set→ Clique

SAT → 3SAT

3SAT→ 3D Match

3D Match→ ZOE

ZOE→ Rudrata

Rudrata→ TSPAll Problems in NP→SAT

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 21

Rudrata (s, t)-Path→ Rudrata cycle

Hard Problems

Hard Problems

Search Problems

Complexity Class

Reductions

Reductions

Reductions

TSP→ TSP with budgetb

⊲Rudrata(s, t)-Path→ Rudrata cycle

3SAT→ Independent SetIndependent Set→ VertexCover

Independent Set→ Clique

SAT → 3SAT

3SAT→ 3D Match

3D Match→ ZOE

ZOE→ Rudrata

Rudrata→ TSPAll Problems in NP→SAT

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 22

� Is Rudrata cycle harder than Rudrata(s, t)-Path? No. Because ...

3SAT→ Independent Set

Hard Problems

Hard Problems

Search Problems

Complexity Class

Reductions

Reductions

Reductions

TSP→ TSP with budgetb

Rudrata(s, t)-Path→Rudrata cycle

⊲3SAT→ IndependentSet

Independent Set→ VertexCover

Independent Set→ Clique

SAT → 3SAT

3SAT→ 3D Match

3D Match→ ZOE

ZOE→ Rudrata

Rudrata→ TSPAll Problems in NP→SAT

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 23

Independent Set→ Vertex Cover

Hard Problems

Hard Problems

Search Problems

Complexity Class

Reductions

Reductions

Reductions

TSP→ TSP with budgetb

Rudrata(s, t)-Path→Rudrata cycle

3SAT→ Independent Set

⊲Independent Set→Vertex Cover

Independent Set→ Clique

SAT → 3SAT

3SAT→ 3D Match

3D Match→ ZOE

ZOE→ Rudrata

Rudrata→ TSPAll Problems in NP→SAT

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 24

Independent Set→ Clique

Hard Problems

Hard Problems

Search Problems

Complexity Class

Reductions

Reductions

Reductions

TSP→ TSP with budgetb

Rudrata(s, t)-Path→Rudrata cycle

3SAT→ Independent SetIndependent Set→ VertexCover

⊲Independent Set→Clique

SAT → 3SAT

3SAT→ 3D Match

3D Match→ ZOE

ZOE→ Rudrata

Rudrata→ TSPAll Problems in NP→SAT

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 25

SAT → 3SAT

Hard Problems

Hard Problems

Search Problems

Complexity Class

Reductions

Reductions

Reductions

TSP→ TSP with budgetb

Rudrata(s, t)-Path→Rudrata cycle

3SAT→ Independent SetIndependent Set→ VertexCover

Independent Set→ Clique

⊲ SAT → 3SAT

3SAT→ 3D Match

3D Match→ ZOE

ZOE→ Rudrata

Rudrata→ TSPAll Problems in NP→SAT

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 26

3SAT→ 3D Match

Hard Problems

Hard Problems

Search Problems

Complexity Class

Reductions

Reductions

Reductions

TSP→ TSP with budgetb

Rudrata(s, t)-Path→Rudrata cycle

3SAT→ Independent SetIndependent Set→ VertexCover

Independent Set→ Clique

SAT → 3SAT

⊲ 3SAT→ 3D Match

3D Match→ ZOE

ZOE→ Rudrata

Rudrata→ TSPAll Problems in NP→SAT

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 27

3D Match → ZOE

Hard Problems

Hard Problems

Search Problems

Complexity Class

Reductions

Reductions

Reductions

TSP→ TSP with budgetb

Rudrata(s, t)-Path→Rudrata cycle

3SAT→ Independent SetIndependent Set→ VertexCover

Independent Set→ Clique

SAT → 3SAT

3SAT→ 3D Match

⊲ 3D Match→ ZOE

ZOE→ Rudrata

Rudrata→ TSPAll Problems in NP→SAT

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 28

ZOE → Rudrata

Hard Problems

Hard Problems

Search Problems

Complexity Class

Reductions

Reductions

Reductions

TSP→ TSP with budgetb

Rudrata(s, t)-Path→Rudrata cycle

3SAT→ Independent SetIndependent Set→ VertexCover

Independent Set→ Clique

SAT → 3SAT

3SAT→ 3D Match

3D Match→ ZOE

⊲ ZOE→ Rudrata

Rudrata→ TSPAll Problems in NP→SAT

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 29

Rudrata → TSP

Hard Problems

Hard Problems

Search Problems

Complexity Class

Reductions

Reductions

Reductions

TSP→ TSP with budgetb

Rudrata(s, t)-Path→Rudrata cycle

3SAT→ Independent SetIndependent Set→ VertexCover

Independent Set→ Clique

SAT → 3SAT

3SAT→ 3D Match

3D Match→ ZOE

ZOE→ Rudrata

⊲ Rudrata→ TSPAll Problems in NP→SAT

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 30

All Problems in NP → SAT

Hard Problems

Hard Problems

Search Problems

Complexity Class

Reductions

Reductions

Reductions

TSP→ TSP with budgetb

Rudrata(s, t)-Path→Rudrata cycle

3SAT→ Independent SetIndependent Set→ VertexCover

Independent Set→ Clique

SAT → 3SAT

3SAT→ 3D Match

3D Match→ ZOE

ZOE→ Rudrata

Rudrata→ TSP

⊲All Problems in NP→SAT

Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 31