cs483 analysis of algorithms lecture 11 – np …jmlien/teaching/09_spring_cs483/...analysis of...
TRANSCRIPT
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 1
CS483 Analysis of AlgorithmsLecture 11 – NP-completeness∗
Jyh-Ming Lien
April 23, 2009
∗this lecture note is based onAlgorithmsby S. Dasgupta, C.H. Papadimitriou, and U.V. Vazirani andIntro-duction to the Design and Analysis of Algorithmsby Anany Levitin.
Hard Problems
⊲ Hard Problems
Hard Problems
Search Problems
Complexity Class
Reductions
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 2
� Search problems (formal definition later):
– Search for shortest path in a graph– Search for values ofx, y andz to satisfy:x2y
12 z − xz2 = 7
– Search for a path in 3D space among obstacles– ...
� Success on solving these hard problems in polynomial time
– Greedyproperties: without looking backward or forward, MST,shortest paths
– Optimality fromsubproblems: divide and conquer, dynamicprogramming
– Convexity: gradient decent/hill climbing, linear programming
� Success on solving these hard problems is based on somespecialproperties of the problems
Hard Problems
Hard Problems
⊲ Hard Problems
Search Problems
Complexity Class
Reductions
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 3
� Failures
– Many problems require2n, n! or evennn (intractable)– For some (clearly formalized) problems we don’t even have
algorithms to solve them– (not to mention those problems that we can not even formalize)
� In this lecture, we will look at
– What are theseproblems? (Search Problems)– Terminology to classifyproblems: P vs. NP– How do you know if aproblemcan be solved efficiently?
(Problem Reduction)– Do we have any hope of solving theseproblemsefficiently?
Search Problems
Hard Problems
Hard Problems
⊲ Search ProblemsWhat Are SearchProblems?
Optimization Problems
Search Problems
Search Problems
Search Problems
Search Problems
Search Problems
Search Problems
Complexity Class
Reductions
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 4
What Are Search Problems?
Hard Problems
Hard Problems
Search Problems
⊲What Are SearchProblems?
Optimization Problems
Search Problems
Search Problems
Search Problems
Search Problems
Search Problems
Search Problems
Complexity Class
Reductions
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 5
� A Search problem has
– An instance of problemI that is input data specifying the problem– Asked to find a solutionS that meets a particular specification– Polynomial-time Checkable: There most be an algorithmC that
takesI andS and checks for correctnessefficiently, i.e., inpolynomial time
� Example: Satisfability problem
– I : (x ∨ y ∨ z)(x ∨ y)– S : {x = T, y = F, z = T}– C : (T ∨ F ∨ T )(T ∨ F )
� Example: Traveling salesman problem
– I : G = {V, E}– S : {vi, vk, · · · , vi}– C :
Optimization Problems
Hard Problems
Hard Problems
Search ProblemsWhat Are SearchProblems?
⊲ Optimization Problems
Search Problems
Search Problems
Search Problems
Search Problems
Search Problems
Search Problems
Complexity Class
Reductions
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 6
� We convert an optimization problem to a search problem
– by introduce abudgeb
– Budget does not make the problem harder or easier.
� Example: Traveling salesman problem with budgetb
– I : G = {V, E}, b– S : {vi, vk, · · · , vi}– C :
� Why do we convert an optimization problem to a search problem?
Search Problems
Hard Problems
Hard Problems
Search ProblemsWhat Are SearchProblems?
Optimization Problems
⊲ Search Problems
Search Problems
Search Problems
Search Problems
Search Problems
Search Problems
Complexity Class
Reductions
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 7
� Many problems we studied in the previous chapters are searchproblems, e.g., all-pairs shortest paths problem, single-source shortestpaths problem, minimum spanning tree, maximum flow minimum cut,matching, ...
– But why are these problem tractable? These problems seem to haveVeryLargesearch spaces
– Many algorithms seem to defeat the curse of expoentiality!
� Now, after we have seen the most brilliant successes, it’s about timefor us to face some failure in this quest.
– Satisfability (SAT, 2SAT, 3SAT)– MST and TSP (traveling salesman problem) with or without budgetb– Euler and Rudrata– Minimum cuts and balanced cuts– Integer linear programming or ILP (Ax ≤ b) and Zero-one Equations or
ZOE (Ax = 1)– Three dimensional matching– Independent set, vertex cover, clique problem (with budget b)– Longest path
Search Problems
Hard Problems
Hard Problems
Search ProblemsWhat Are SearchProblems?
Optimization Problems
Search Problems
⊲ Search Problems
Search Problems
Search Problems
Search Problems
Search Problems
Complexity Class
Reductions
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 8
� Satisfability problems (Horn-SAT, 2SAT, 3SAT, KSAT)� Horn’s formula
– Implications:(z ∧ w) ⇒ u
– Pure negative clauses:(u ∨ v ∨ y)– Horn-SAT: Solvable using greedy algorithm
� 2SAT
– in conjunctive normal form (CNF)– Each clause has two literals– example:(x ∨ y) ∧ (x ∨ z) ∧ (x ∨ w)– Can be solve in polynomail time
� 3SAT
– in conjunctive normal form (CNF)– Each clause has 3 literals– example:(x ∨ y ∨ w) ∧ (x ∨ z ∨ y) ∧ (x ∨ u ∨ w)– No polynomial time algorithm
Search Problems
Hard Problems
Hard Problems
Search ProblemsWhat Are SearchProblems?
Optimization Problems
Search Problems
Search Problems
⊲ Search Problems
Search Problems
Search Problems
Search Problems
Complexity Class
Reductions
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 9
� Euler’s tour and Rudrata’s problem
Euler’s tourvisit all edges without repeatingSolvable in polynomial time
Rudrata’s problemvisit all vertices without repeatingNo polynomial time algorithm
Search Problems
Hard Problems
Hard Problems
Search ProblemsWhat Are SearchProblems?
Optimization Problems
Search Problems
Search Problems
Search Problems
⊲ Search Problems
Search Problems
Search Problems
Complexity Class
Reductions
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 10
� Longest path (Taxicab rip-off prob-lem)
– Give a graph and two nodess andt, find a path with length at leastb
from s to t without repeating ver-tices.
– no polynomial time algorithm
� Minimum cuts and balanced cuts
– Find cuts that split the graph into twosetsS andT
– Minimum cuts problem can be solvedusing linear programming
– Balanced cuts:|S| ≥ n/3 |T | ≥n/3 and there are at mostb edges be-tweenS andT
– Balanced cuts problem has no poly-nomial time algorithm
Search Problems
Hard Problems
Hard Problems
Search ProblemsWhat Are SearchProblems?
Optimization Problems
Search Problems
Search Problems
Search Problems
Search Problems
⊲ Search Problems
Search Problems
Complexity Class
Reductions
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 11
� Three dimensional matching
� Independent set, vertex cover, clique problem (with budgetb)
– Independent set: A set ofb vertices that are not adjacent to eachother
– vertex cover: A set ofb vertices that are incident to all edges– clique: A set ofb vertices that have all possible connections
Search Problems
Hard Problems
Hard Problems
Search ProblemsWhat Are SearchProblems?
Optimization Problems
Search Problems
Search Problems
Search Problems
Search Problems
Search Problems
⊲ Search Problems
Complexity Class
Reductions
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 12
� Knapsack problem and Subset sum
– In subset sum, each item has same value and weight– Both problems have no polynomial time algorithms
� Integer linear programming or ILP (Ax ≤ b) and Zero-one Equationsor ZOE (Ax = 1)
– Simplex method is not polynomial but LP can be solved inpolynomial time
– ILP requires the values of all variables to be integer– ZOE is a special type of ILP where all values inA are 0 or 1– Both problems have no polynomial time algorithms
Complexity Class
Hard Problems
Hard Problems
Search Problems
⊲ Complexity Class
Problem Complexity
P vs. NPP vs. NP vs. NP hard vs.NP completeP vs. NP vs. NP hard vs.NP complete
Reductions
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 13
Problem Complexity
Hard Problems
Hard Problems
Search Problems
Complexity Class
⊲ Problem Complexity
P vs. NPP vs. NP vs. NP hard vs.NP completeP vs. NP vs. NP hard vs.NP complete
Reductions
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 14
� Tractable: a problem is tractable if there is an algorithm can solve theproblem deterministically inpolynomial time
� Is a problem tractable or intractable?
– yes (given an algorithm to support this answer)– no
⊲ because it’s been proved that no algorithm exists at all (e.g.,Turing’sHalting Problem.)
⊲ because it’s been be proved that any algorithm takesexponential time (Traveling Sales Men Problem)
– we have no idea...
P vs. NP
Hard Problems
Hard Problems
Search Problems
Complexity Class
Problem Complexity
⊲ P vs. NPP vs. NP vs. NP hard vs.NP completeP vs. NP vs. NP hard vs.NP complete
Reductions
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 15
� Hard problems, easy problemsHard problemsNP-complete Easy problemsP
SAT, 3SAT 2SATTraveling Salesman Problem, Rudrata pathChinese Postman Problem, Euler path
3D matching Bipartite matchingIndependent set Independent set on trees
Integer linear programming Linear programmingBalance cut Minimum cut
� P: polynomial
– Given an instanceI, we can find a polynomial time algorithm to find an solutionS
� NP: nondeterministic polynomial
– Given an instanceI and a proposed solutionS, we can find a polynomial timealgorithmC to check ifS is an solution ofI
– Remember: A problem inNP doesNOT mean it is a hard problem
� NP hard: all problems in NP can bereducedto a NP hard problem
– A NP hard problem is at least as hard as the hardest problem inNP
� NP complete: in NP and also in NP hard
P vs. NP vs. NP hard vs. NP complete
Hard Problems
Hard Problems
Search Problems
Complexity Class
Problem Complexity
P vs. NP
⊲P vs. NP vs. NP hardvs. NP complete
P vs. NP vs. NP hard vs.NP complete
Reductions
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 16
� Their relationship
P vs. NP vs. NP hard vs. NP complete
Hard Problems
Hard Problems
Search Problems
Complexity Class
Problem Complexity
P vs. NPP vs. NP vs. NP hard vs.NP complete
⊲P vs. NP vs. NP hardvs. NP complete
Reductions
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 17
� Problems in Psorting, MST, ...
� Problems in NP completeSAT, TSP, ILP, ...
� Problems in NP but not in P or in NP completefactoring, graph isomorphism
� Problems not in NPHalting problem, counting the number of perfect matching, matrix permanent,...
� P=NP? (Most Computer Scientists believe P6= NP)� There are many more complexity classes than these three (PSpace, Co-NP,
ExpTime, ExpSpace,...)In fact, there are 462 complexity classes according to the “Complexity zoo”http://qwiki.caltech.edu/wiki/ComplexityZoo (maintained by Scott Aaronson and GregKuperberg)
Reductions
Hard Problems
Hard Problems
Search Problems
Complexity Class
⊲ Reductions
Reductions
Reductions
TSP→ TSP with budgetb
Rudrata(s, t)-Path→Rudrata cycle
3SAT→ Independent SetIndependent Set→ VertexCover
Independent Set→ Clique
SAT → 3SAT
3SAT→ 3D Match
3D Match→ ZOE
ZOE→ Rudrata
Rudrata→ TSPAll Problems in NP→SAT
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 18
Reductions
Hard Problems
Hard Problems
Search Problems
Complexity Class
Reductions
⊲ Reductions
Reductions
TSP→ TSP with budgetb
Rudrata(s, t)-Path→Rudrata cycle
3SAT→ Independent SetIndependent Set→ VertexCover
Independent Set→ Clique
SAT → 3SAT
3SAT→ 3D Match
3D Match→ ZOE
ZOE→ Rudrata
Rudrata→ TSPAll Problems in NP→SAT
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 19
� If we reduce a problemA to a problemB in polynomial time (denoted asA → B, we can say thatB is as hard asA if not harder
– If A → B andA is NP-complete then we know thatB is alsoNP-complete
Reductions
Hard Problems
Hard Problems
Search Problems
Complexity Class
Reductions
Reductions
⊲ Reductions
TSP→ TSP with budgetb
Rudrata(s, t)-Path→Rudrata cycle
3SAT→ Independent SetIndependent Set→ VertexCover
Independent Set→ Clique
SAT → 3SAT
3SAT→ 3D Match
3D Match→ ZOE
ZOE→ Rudrata
Rudrata→ TSPAll Problems in NP→SAT
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 20
� Reductions between NP-complete problems
TSP→ TSP with budget b
Hard Problems
Hard Problems
Search Problems
Complexity Class
Reductions
Reductions
Reductions
⊲TSP→ TSP withbudgetb
Rudrata(s, t)-Path→Rudrata cycle
3SAT→ Independent SetIndependent Set→ VertexCover
Independent Set→ Clique
SAT → 3SAT
3SAT→ 3D Match
3D Match→ ZOE
ZOE→ Rudrata
Rudrata→ TSPAll Problems in NP→SAT
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 21
Rudrata (s, t)-Path→ Rudrata cycle
Hard Problems
Hard Problems
Search Problems
Complexity Class
Reductions
Reductions
Reductions
TSP→ TSP with budgetb
⊲Rudrata(s, t)-Path→ Rudrata cycle
3SAT→ Independent SetIndependent Set→ VertexCover
Independent Set→ Clique
SAT → 3SAT
3SAT→ 3D Match
3D Match→ ZOE
ZOE→ Rudrata
Rudrata→ TSPAll Problems in NP→SAT
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 22
� Is Rudrata cycle harder than Rudrata(s, t)-Path? No. Because ...
3SAT→ Independent Set
Hard Problems
Hard Problems
Search Problems
Complexity Class
Reductions
Reductions
Reductions
TSP→ TSP with budgetb
Rudrata(s, t)-Path→Rudrata cycle
⊲3SAT→ IndependentSet
Independent Set→ VertexCover
Independent Set→ Clique
SAT → 3SAT
3SAT→ 3D Match
3D Match→ ZOE
ZOE→ Rudrata
Rudrata→ TSPAll Problems in NP→SAT
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 23
Independent Set→ Vertex Cover
Hard Problems
Hard Problems
Search Problems
Complexity Class
Reductions
Reductions
Reductions
TSP→ TSP with budgetb
Rudrata(s, t)-Path→Rudrata cycle
3SAT→ Independent Set
⊲Independent Set→Vertex Cover
Independent Set→ Clique
SAT → 3SAT
3SAT→ 3D Match
3D Match→ ZOE
ZOE→ Rudrata
Rudrata→ TSPAll Problems in NP→SAT
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 24
Independent Set→ Clique
Hard Problems
Hard Problems
Search Problems
Complexity Class
Reductions
Reductions
Reductions
TSP→ TSP with budgetb
Rudrata(s, t)-Path→Rudrata cycle
3SAT→ Independent SetIndependent Set→ VertexCover
⊲Independent Set→Clique
SAT → 3SAT
3SAT→ 3D Match
3D Match→ ZOE
ZOE→ Rudrata
Rudrata→ TSPAll Problems in NP→SAT
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 25
SAT → 3SAT
Hard Problems
Hard Problems
Search Problems
Complexity Class
Reductions
Reductions
Reductions
TSP→ TSP with budgetb
Rudrata(s, t)-Path→Rudrata cycle
3SAT→ Independent SetIndependent Set→ VertexCover
Independent Set→ Clique
⊲ SAT → 3SAT
3SAT→ 3D Match
3D Match→ ZOE
ZOE→ Rudrata
Rudrata→ TSPAll Problems in NP→SAT
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 26
3SAT→ 3D Match
Hard Problems
Hard Problems
Search Problems
Complexity Class
Reductions
Reductions
Reductions
TSP→ TSP with budgetb
Rudrata(s, t)-Path→Rudrata cycle
3SAT→ Independent SetIndependent Set→ VertexCover
Independent Set→ Clique
SAT → 3SAT
⊲ 3SAT→ 3D Match
3D Match→ ZOE
ZOE→ Rudrata
Rudrata→ TSPAll Problems in NP→SAT
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 27
3D Match → ZOE
Hard Problems
Hard Problems
Search Problems
Complexity Class
Reductions
Reductions
Reductions
TSP→ TSP with budgetb
Rudrata(s, t)-Path→Rudrata cycle
3SAT→ Independent SetIndependent Set→ VertexCover
Independent Set→ Clique
SAT → 3SAT
3SAT→ 3D Match
⊲ 3D Match→ ZOE
ZOE→ Rudrata
Rudrata→ TSPAll Problems in NP→SAT
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 28
ZOE → Rudrata
Hard Problems
Hard Problems
Search Problems
Complexity Class
Reductions
Reductions
Reductions
TSP→ TSP with budgetb
Rudrata(s, t)-Path→Rudrata cycle
3SAT→ Independent SetIndependent Set→ VertexCover
Independent Set→ Clique
SAT → 3SAT
3SAT→ 3D Match
3D Match→ ZOE
⊲ ZOE→ Rudrata
Rudrata→ TSPAll Problems in NP→SAT
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 29
Rudrata → TSP
Hard Problems
Hard Problems
Search Problems
Complexity Class
Reductions
Reductions
Reductions
TSP→ TSP with budgetb
Rudrata(s, t)-Path→Rudrata cycle
3SAT→ Independent SetIndependent Set→ VertexCover
Independent Set→ Clique
SAT → 3SAT
3SAT→ 3D Match
3D Match→ ZOE
ZOE→ Rudrata
⊲ Rudrata→ TSPAll Problems in NP→SAT
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 30
All Problems in NP → SAT
Hard Problems
Hard Problems
Search Problems
Complexity Class
Reductions
Reductions
Reductions
TSP→ TSP with budgetb
Rudrata(s, t)-Path→Rudrata cycle
3SAT→ Independent SetIndependent Set→ VertexCover
Independent Set→ Clique
SAT → 3SAT
3SAT→ 3D Match
3D Match→ ZOE
ZOE→ Rudrata
Rudrata→ TSP
⊲All Problems in NP→SAT
Analysis of Algorithms CS483 Lecture 11 – NP-completeness trans – 31