cse 024: design & analysis of algorithms chapter 9: np completeness sedgewick chp:40 david...

CSE 024: Design & Analysis of

Algorithms

Chapter 9: NP CompletenessSedgewick Chp:40

David Luebke’s Course Notes / University of Virginia, Computer Science

April 21, 2023 CS 615 Design & Analysis of Algorithms 2

Course Content

1. Introduction, Algorithmic Notation and Flowcharts (Brassard & Bratley, Chap. 3)

2. Efficiency of Algorithms (Brassard & Bratley, Chap. 2)

3. Basic Data Structures (Brassard & Bratley, Chap. 5)4. Sorting (Weiss, Chap. 7)5. Searching (Brassard & Bratley, Chap. 9)6. Graph Algorithms (Weiss, Chap. 9)

7.7. Randomized Algorithms (WeissRandomized Algorithms (Weiss,, Chap Chap.. 10) 10)8. String Searching (Sedgewick, Chap. 19)9. NP Completeness (Sedgewick, Chap. 40)


Lecture Content

IntractabilityP & NP ProblemsHamiltonian Path ProblemReductionNP-Hard and NP-Complete ProblemsThe Traveling Salesman ProblemConjunctive Normal FormClique & Vertex CoverGeneral Comments

TractibilitySome problems are undecidable:

no computer can solve them

E.g., Turing’s “Halting Problem”take an Abstract Machines & Languages class

Other problems are decidable, but intractable:

as they grow large, we are unable to solve them in reasonable time

What constitutes reasonable time?Polynomial algorithms!


Efficiency of Algorithms:polynomial time / non-polynomial time


O(1)O(n logn)

O(n2)

O(n3)

O(n10)

O(nlog n)

O(2n)

O(n!)

O(222n)

polynomial time

O(nc) for some constant c

non-polynomialtime

Polynomial Time AlgorithmsAre there problems

solvable in polynomial time?Answer is yes:

every algorithm we’ve studied provides polynomial-time solution to some problems

Define P:the class of problems solvable in polynomial time

Can we solve all problems that are solvable in polynomial time?

No: Turing’s “Halting Problem” is not solvable by any computer no matter how much time is given

Such problems are clearly intractable, not in PApril 21, 2023 CS 615 Design & Analysis of Algorithms 6

Turing’s Halting ProblemDefinition of algorithm

a procedure always terminates and gives an answer.

Formal Definition of Algorithma Turing machine that always halts whether succeeds or fails

A problem is solvable if there is an algorithm solves every instance of the given problem

A problem is said to be unsolvable if no algorithm exists to solve the given problem

The Halting Problem for Turing machines:“Does there exist an effective procedure

for a computable function e(M)to decide for every pair (e(M),x);

does M halt for x?”The Halting problem for Turing machine is unsolvable!If it were solvable

There is a Turing Machine that can solve the halting problemThere is another Turing Machine to solve the problem above...


P and NPP (Polynomial Time)

the set of problems can be solved in polynomial time

NP (Nondeterministic Polynomial Time)

provably decidable in polynomial time on a nondeterministic computer

P NPThe big question:

Does P = NP?


Determinist-Nondeterminist Machines


Nondeterministic Finite State Automata Example

Deterministic Finite State Automata Example

ReductionA problem P can be reduced

to another problem Q if any instance of P can be rephrased to an instance of Q,

This rephrasing is called a transformation

Intuitively: If P reduces in polynomial time to Q, P is “no harder to solve” than Q


Reducibility: ExampleP: Given a set of Booleans,

is at least one TRUE?

Q: Given a set of integers, is their sum positive?

Transformation: (x1, x2, …, xn) = (y1, y2, …, yn)

where yi = 1 if xi = TRUE,

yi = 0 if xi = FALSE


NP-Hard and NP-CompleteP p Q

P is polynomial-time reducible to Q,

If all problems R NP are reducible to P, then P is NP-Hard

We say P is NP-Complete if P is NP-Hard and P NP

If P p Q and P is NP-Complete,

Q is also NP- Complete


Using ReductionsIf P is polynomial-time reducible to Q,

we denote this P p Q

Definition of NP-Complete: If P is NP-Complete,

P NP and all problems R are reducible to P

Formally: R p P R NP

If P p Q and P is NP-Complete,

Q is also NP-Complete


Popular NP-Complete ProblemsGiven one NP-Complete problem,

we can prove many interesting problems are NP-Complete

Popular Problems:Boolean satisfiability problem (Sat.)N-puzzleKnapsack problemHamiltonian path problemTravelling salesman problemSubgraph isomorphism problemSubset sum problemClique problemVertex cover problemIndependent set problemDominating set problemGraph coloring problem


NP-Complete ProblemsNP-Complete problems

are the “hardest” problems in NP:

If any NP-Complete problem can be solved in polynomial time

then every NP-Complete problems can be solved in polynomial time

To show every problem in NP solve hamiltonian-cycle in O(n100) time Then it is proved that P = NP


Hamiltonian Cycle & GraphGiven a graph G=(V,E)Find a path

To visit all nodeswithout visiting any nodes twice!

A Hamiltonian cycle, also: Hamiltonian circuit, vertex tour or graph cycle

a cycle that visits each vertex exactly once except the vertex which is both the start and end,

and so is visited twice

A graph that contains a Hamiltonian cycle is called a Hamiltonian graph.


The Traveling Salesman ProblemThe well-known traveling salesman problem:

a salesman must travel to n cities, visiting each city exactly once

and finishing where he beginswith minimum cost!

Model as complete graph with cost c(i,j)

to go from city i to city j

How would we turn this into a decision problem?

Is there a simple cycle over cities with cost < k ?

ask if a TSP with cost < k April 21, 2023 CS 615 Design & Analysis of Algorithms 17

An optimal TSP tour through Germany’s 15 largest cities. It is the shortest among 43 589 145 600 possible tours visiting each city exactly once

Hamiltonian Cycle TSPHow can we prove the TSP is NP-Complete?A: Prove TSP NP;

reduce the undirected hamiltonian cycle problem to TSP

TSP NP: straightforwardReduction: need to show that

if we can solve TSP we can solve hamiltonian cycle problem


The Satisfiability (SAT) ProblemOne of the first problems

to be proved NP-Complete was satisfiability (SAT):

Given a Boolean expression on n variables,

can we assign values such that the expression is TRUE?

Ex: ((x1 x2) ((x1 x3) x4)) x2

Cook’s Theorem: The satisfiability problem is NP-Complete


Conjunctive Normal FormA Boolean formula is

in conjunctive normal form, or CNF, if it is an AND of clauses,

each of which is an OR of literals

Ex: (x1 x2) (x1 x3 x4) (x5)

3-CNF: each clause has exactly 3 distinct literalsEx: (x1 x2 x3) (x1 x3 x4) (x5 x3 x4)

Result is true if at least one literal in each clause is true


The 3-CNF ProblemTheorem 36.10:

Satisfiability of Boolean formulas in 3-CNF form is NP-CompleteProof: Nope

the 3-CNF problem is it is relatively easy to reduce to others

Thus by proving 3-CNF NP-Complete we can prove many seemingly unrelated problemsNP-Complete


3-CNF CliqueClique of a graph G

a subset of vertices fully connected to each otheri.e. a complete subgraph of G

The clique problem: how large is the maximum-size clique in a graph?

k-clique problemcan we turn clique problem

into a decision problem?

Is the k-clique problem within NP?


A graph with 23 1-vertex cliques (its vertices), 42 2-vertex cliques (its edges), 19 3-vertex cliques (the light blue triangles), 2 4-vertex cliques (dark blue).

Clique Vertex CoverA vertex cover for a graph G

is a set of vertices incident to every edge in G

The vertex cover problem: what is the minimum size vertex cover in G?

Restated as a decision problem: does a vertex cover of size k exist in G?

Thm 36.12: vertex cover is NP-Complete


Clique Vertex CoverFirst, show vertex cover in NPNext, reduce k-clique to vertex cover

The complement GC of a graph G contains exactly those edges not in G

Compute GC in polynomial time

G has a clique of size k iff GC has a vertex cover of size |V| - k


Clique Vertex CoverClaim:

If G has a clique of size k, GC has a vertex cover of size |V| - k

Let V’ be the k-cliqueThen V - V’ is a vertex cover in GC

Let (u,v) be any edge in GC

Then u and v cannot both be in V’Why?

Thus at least one of u or v is in V-V’ Why?So edge (u, v) is covered by V-V’

Since true for any edge in GC, V-V’ is a vertex cover


Clique Vertex CoverClaim:

If GC has a vertex cover V’ V, with |V’| = |V| - k,

then G has a clique of size kFor all u,v V,

if (u,v) GC then u V’ or v V’ or both Why?

Contrapositive: if u V’ and v V’, Then (u,v) E

In other words, all vertices in V-V’

are connected by an edge, thus V-V’ is a clique

Since |V| - |V’| = k, the size of the clique is k


General CommentsLiterally hundreds of problems

have been shown to be NP-Complete

Some reductions are profound, some are comparatively easy, many are easy once the key insight is given


Some NP-Complete ProblemsSubset-sum:

Given a set of integers, does there exist a subset

adds up to some target T ?

0-1 knapsack: when weights not just integers

Hamiltonian path: ObviousGraph coloring:

can a given graph be colored with k colors

such that no adjacent vertices are the same color?

Etc…



Chapter 9 NP Completeness

cse 024: design & analysis of algorithms chapter 9: np completeness sedgewick chp:40 david...

Documents