11

The The TaleTale of ofNP-CompletenessNP-Completeness

Tal KramerTal Kramer

Gillat KolGillat Kol

22

Hilbert’s Three QuestionsHilbert’s Three QuestionsIn 1900 In 1900 DavidDavid HilbertHilbert asks 3 questions:asks 3 questions:

1. 1. Is mathematics Is mathematics completecomplete??  (can  (can everyevery mathematical statement be either proved mathematical statement be either proved or disproved?) or disproved?)

2. 2. Is mathematics Is mathematics consistentconsistent??  (is it true that  (is it true that statements such as 0 = 1 cannot be statements such as 0 = 1 cannot be proved by valid methods?) proved by valid methods?)

3. 3. Is mathematics Is mathematics decidabledecidable?? (The mighty (The mighty

EntscheidungsproblemEntscheidungsproblem)) (is there a 'mechanical' (is there a 'mechanical' method that can be applied to method that can be applied to anyany mathematical assertion in order to check mathematical assertion in order to check whether that assertion is true or not?)whether that assertion is true or not?)

33

Hilbert’s Three NOsHilbert’s Three NOsHilbertHilbert gets 3 NOs: gets 3 NOs:

19311931 – – Kurt GödelKurt Gödel introduces the incompleteness introduces the incompleteness theorems theorems

19361936 – – AlonzoAlonzo ChurchChurch proves that there is no proves that there is no algorithm which decides for two given lambda algorithm which decides for two given lambda calculus expressions whether they are equivalent. calculus expressions whether they are equivalent.

19361936 – – AlanAlan TuringTuring “builds” a-machines (a.k.a “builds” a-machines (a.k.a Turing Turing MachinesMachines) and reduces the halting problem for ) and reduces the halting problem for TMTMs to the s to the EntscheidungsproblemEntscheidungsproblem..

TMs became a platform for general computationsTMs became a platform for general computations..

44

The Birth of Computational The Birth of Computational ComplexityComplexity

The idea of “The idea of “efficient algorithmsefficient algorithms" can " can be traced back to the ancient be traced back to the ancient Greeks.Greeks.

But our story starts in the 1960’s when But our story starts in the 1960’s when Juris HartmanisJuris Hartmanis and and Richard E. Richard E. StearnsStearns, laid out the definition of , laid out the definition of quantified time and space quantified time and space complexity, and thus the complexity, and thus the computational complexity was borncomputational complexity was born

55

The Mother of PThe Mother of P

In 1965, In 1965, JackJack EdmondsEdmonds decides to look for decides to look for an an efficient algorithmefficient algorithm for for maximal maximal matchingmatching, even though a finite , even though a finite algorithm was known:algorithm was known:

““It may be that since one is customarily It may be that since one is customarily concerned with existence, convergence, concerned with existence, convergence, finiteness, and so forth, one is not finiteness, and so forth, one is not inclined to take seriously the question inclined to take seriously the question of existence of a better-than-finite of existence of a better-than-finite algorithm”algorithm”

P

66

The Birth of PThe Birth of P

EdmondsEdmonds defines defines asymptotic estimateasymptotic estimate (“(“OrderOrder”) of an algorithm running ”) of an algorithm running time as a measure of “time as a measure of “how goodhow good” ” the algorithm is and defines the the algorithm is and defines the class class PP..

““For practical purposes, the difference For practical purposes, the difference between algebraic and exponential between algebraic and exponential order is often more crucial than the order is often more crucial than the difference between finite and difference between finite and infinite.”infinite.”

77

First Formal Proof that P First Formal Proof that P NP NP

By giving a poly-time solution for the By giving a poly-time solution for the matching search problem, matching search problem, EdmondsEdmonds initiates the search for more such initiates the search for more such algorithms. Though, he himself guesses algorithms. Though, he himself guesses that there are some problems that doesn’t that there are some problems that doesn’t have an efficient solution: have an efficient solution:

““I conjecture that there are no good algorithm I conjecture that there are no good algorithm for the Traveling Salesman Problem. My for the Traveling Salesman Problem. My reasons are the same as for any reasons are the same as for any mathematical conjecture mathematical conjecture

(1) It is a legitimate mathematical possibility(1) It is a legitimate mathematical possibility

(2) I don’t Know.” (1966)(2) I don’t Know.” (1966)

88

Gödel’s Letter to von Neumann Gödel’s Letter to von Neumann (1956)(1956)

Settings: Settings:

MM == TMTM that for every first order formula that for every first order formula F F and an input and an input nn checks if checks if FF has a proof of has a proof of length length nn..

ΨΨMM(F,n)(F,n) == The running time of The running time of MM on on FF..

MM(n) = max(n) = maxF F { { ΨΨMM(F,n) } (F,n) }

A Riddle:A Riddle: IsIs there anthere an M M s.t.s.t. MM (n) ~ k·n (n) ~ k·n (or at most(or at most

k·nk·n22)? ()? (GödelGödel guesses that it’s true) guesses that it’s true)

99

Gödel’s Letter – cont.Gödel’s Letter – cont.Importance:Importance:

““It would obviously mean that in spite of It would obviously mean that in spite of the undecidability of the the undecidability of the EntscheidungsproblemEntscheidungsproblem, , the mental work the mental work of a mathematician concerning Yes-or-No of a mathematician concerning Yes-or-No questions could be completely replaced questions could be completely replaced by a machineby a machine. After all, one would simply . After all, one would simply have to choose the natural number have to choose the natural number nn so so large that when the machine does not large that when the machine does not deliver a result, it makes no sense to deliver a result, it makes no sense to think more about the problemthink more about the problem..””

1010

Cook Cook (1971)(1971)

Stephen A. CookStephen A. Cook notices that certain problems notices that certain problems captures the difficulty of the whole captures the difficulty of the whole complexity class, the other problems are complexity class, the other problems are simply easier. To formalize this, the notion simply easier. To formalize this, the notion of a reduction is introduced:of a reduction is introduced:

Def:Def: An An oracleoracle for a problem for a problem is a magical is a magical apparatus that, given an input apparatus that, given an input xx to to , , returns returns (x)(x) in a single step. in a single step.

Def:Def: A A Turing (Cook) reductionTuring (Cook) reduction from problem from problem 11 to problem to problem 22 is a poly-oracle machine is a poly-oracle machine that solves that solves 11 on input on input xx while getting oracle while getting oracle answers for problem answers for problem 22..

1111

Why Cook? Why Cook?

CookCook shows that any language that is shows that any language that is accepted by some accepted by some NTMNTM within within polynomial time, polynomial time, or that its or that its complement iscomplement is, can be , can be Cook-Cook-reduciblereducible to the sets to the sets CNF-SATCNF-SAT, , DNF-DNF-Tautologies Tautologies (ANDs of Ors), (ANDs of Ors), Subgraph-PairsSubgraph-Pairs, , TautologiesTautologies and and 3-3-DNF-TautologiesDNF-Tautologies..

Well, it sounds a lot like Well, it sounds a lot like NPCNPC……

1212

But, it’s Not Really NPCBut, it’s Not Really NPC

CookCook sees sees NPNP and and coNPcoNP as one class. as one class.

Cook-reductionCook-reduction merges “our” merges “our” NP-NP-completecomplete and and coNP-completecoNP-complete..

coNPC:coNPC:

DNF-TautologiesDNF-Tautologies

TautologiesTautologies

3-DNF-Tautologies3-DNF-Tautologies

NPC:NPC:

Subgraph-PairsSubgraph-Pairs

NP + NP + coNPcoNP

NPC + NPC + coNPCcoNPC Cook

Example: ~CliqueExample: ~CliquecoNP coNP cook CliqueCliqueNPNP

1313

Cook Leaves Some Small Open Cook Leaves Some Small Open QuestionsQuestions

CookCook suggests to prove that: suggests to prove that:

PrimesPrimes and and Graph-IsomorphismGraph-Isomorphism are “ are “NP-NP-CompleteComplete”, but ”, but “It would seem to require “It would seem to require some deep results in number theory”some deep results in number theory”..

TautologiesTautologies is in is in PP, , “Such a proof would be “Such a proof would be a major breakthrough in complexity a major breakthrough in complexity theory”theory”..

In The small issue of P vs.

NP

Unfortunately in P

Still Open

1414

After Cook – There was Karp After Cook – There was Karp (1972)(1972)

Richard M. KarpRichard M. Karp Formally defines: Formally defines:

The class The class PP as the class of languages as the class of languages recognizable by one-tape poly-time recognizable by one-tape poly-time TMTM. Notes . Notes that the same that the same PP class is obtained using class is obtained using various machinesvarious machines. .

The class The class NPNP as set of languages that as set of languages that have a have a poly-length proof (witness)poly-length proof (witness). Shows that it’s . Shows that it’s the class of languages recognizable by a poly-the class of languages recognizable by a poly-time time NTMNTM ( (CookCook’s notion).’s notion).

““PolynomialPolynomial CompleteComplete” sets (” sets (NPC NPC as we know as we know it).it).

1515

Karp ReductionKarp Reduction

KarpKarp defines a stronger defines a stronger reduction than reduction than CookCook’s, one that can differentiate ’s, one that can differentiate between between NPNP and and coNPcoNP..

Def:Def: A A Karp reductionKarp reduction of of LL11 toto L L22 is a is a polynomial-time computable function polynomial-time computable function ff s.t. s.t.

x x L L1 1 f(x) f(x) L L22

ClaimClaim: : Karp reductionKarp reduction is a special case of is a special case of Cook reductionCook reduction (i.e., if A (i.e., if A≺≺karpkarpB then B then AA≺≺cookcookBB))

1616

Karp’s Complete ProblemsKarp’s Complete Problems

1717

The World by KarpThe World by Karp

PP2-SAT,

Shortest-Path, Minimum-Cut,

Arc-Cover

?? NP-HardNP-Hard

NP-CompleteNP-CompleteSAT, Clique, Hamiltonian-Circuit, Chromatic Number . . .

Equivalence of Regular

Expression,

Equivalence of ND Finite

Automata,

Context Sensitive Recognition

Lin

ear-I

nequ

aliti

es

Graph

-Isom

orph

ism,

Non-P

rimes

NPNP

?? ??

In NPC

In P

1818

Meanwhile, on the Other Side Meanwhile, on the Other Side of the Galaxyof the Galaxy

By the 1950’s the in By the 1950’s the in Soviet UnionSoviet Union people were people were trying to fight off the ‘trying to fight off the ‘perebor problemsperebor problems’ ’ (problems that requires (problems that requires exhaustive-searchexhaustive-search). ). Actually, it was widely thought to be Actually, it was widely thought to be impossible.impossible.

They found problems that can’t possibly be They found problems that can’t possibly be solved without solved without perebor perebor (such as finding a (such as finding a minimal circuit for a Boolean function)minimal circuit for a Boolean function)

One strategy for attacking the One strategy for attacking the perebor problemsperebor problems was the usage of was the usage of probabilistic algorithmsprobabilistic algorithms

1919

Levin Levin (1973)(1973)

In the 1970’sIn the 1970’s Leonid A. Levin Leonid A. Levin deals with the deals with the same issues as same issues as CookCook and and KarpKarp at the same at the same time.time.

Levin Levin main interest lays with themain interest lays with the Sequential Sequential search problems, search problems, (as apposed to (as apposed to CookCook’s and ’s and KarpKarp’s ’s decision problemsdecision problems):): “Given x, find “Given x, find some y of length comparable with the length of some y of length comparable with the length of x such that A(x, y) holds. Where A(x, y) is some x such that A(x, y) holds. Where A(x, y) is some property to be tested by an algorithm whose property to be tested by an algorithm whose operating time is comparable with the length of operating time is comparable with the length of x”x”..

2020

Of course, Of course, LevinLevin defines it’s own reduction: defines it’s own reduction:

Def:Def: a a Levin reductionLevin reduction from from RR11 to to RR22 (relations) is 3 poly-time computable (relations) is 3 poly-time computable functions functions f,g,hf,g,h s.t. s.t.– x x L(R L(R11)) f(x) f(x) L(R L(R22) ) (Karp)(Karp)– (x, y) (x, y) R R11 (f(x), g(x,y)) (f(x), g(x,y)) R R22

– (f(x), z) (f(x), z) R R22 (x, h(x,z)) (x, h(x,z)) R R11

ff translates inputs of the first problem to inputs translates inputs of the first problem to inputs of the second problem,of the second problem, g g & & hh transform transform certificates of one to the other.certificates of one to the other.

Claim:Claim: Levin reductionLevin reduction implies implies Karp Karp reductionreduction for decision problems for decision problems

Levin’s ReductionLevin’s Reduction

2121

Levin – cont.Levin – cont.

Using his reductionUsing his reduction Levin Levin defines defines ““Universal problemsUniversal problems” as problems the ” as problems the all other sequential search problems can all other sequential search problems can be reduced to them, and shows a few be reduced to them, and shows a few examples. examples.

He proves that the existence of a search He proves that the existence of a search problem that is not solvable in poly-time problem that is not solvable in poly-time is enough to show that none of the is enough to show that none of the universal problems universal problems can be solved in can be solved in poly-time.poly-time.

2222

LevinLevin – cont– cont..

LevinLevin proves that for any sequential proves that for any sequential search problem, there exists an search problem, there exists an algorithm algorithm AA that is that is optimaloptimal for all for all xx, up , up to a constant factor and an addition of to a constant factor and an addition of number comparable to (polynomial in) number comparable to (polynomial in) the length of the length of xx..

2323

Why is P vs. NP So Why is P vs. NP So Important? Important?

Money -Money - It’s (literally) a million It’s (literally) a milliondollar questiondollar question

Philophy –Philophy –

- - Is it really more difficult to Is it really more difficult to find a proof than to check it?find a proof than to check it?

-- Is nature nondeterministic?Is nature nondeterministic?

““Real Life” –Real Life” – A Classification tool for what’s A Classification tool for what’s practicalpractical

ComputationalComputational HardnessHardness –– Can be used by the Can be used by the force of good (force of good (PPNPNP is crucial for the existence is crucial for the existence of of one-way-functionsone-way-functions) )

2424

Is it Really THE Question?Is it Really THE Question?Isn’tIsn’t it good enough to approximate \ get a it good enough to approximate \ get a

probabilistic answer?probabilistic answer?

- Although some - Although some NPCNPC problems are easy to problems are easy toapproximate, other approximations are approximate, other approximations are NPCNPC..

- Depends who you ask… - Depends who you ask…

Isn’t it enough to take the average case (as Isn’t it enough to take the average case (as apposed to worst case)? apposed to worst case)?

- For example, existence of a - For example, existence of a one-way-one-way-function function requires a problem that is hard on requires a problem that is hard on average.average.

- Depends who you ask… - Depends who you ask…

2525

Is it a YES\NO Question?

Gödel's First Incompleteness Theorem:Gödel's First Incompleteness Theorem:

For any For any consistentconsistent formal theory formal theory including basic arithmetical truths, including basic arithmetical truths, it it is possible to construct a statement is possible to construct a statement that is true but not included in the that is true but not included in the theorytheory. .

A Lazy Thought:A Lazy Thought:

Can Can P = NPP = NP be one of those be one of those statement?statement?

2626

Ladner’s Theorem Ladner’s Theorem (1975)(1975)

Do P and NPC really divide the world?Do P and NPC really divide the world?

Ladner Theorem:Ladner Theorem: Let Let INP INP NP \ (NPC NP \ (NPC P).P). If If PPNPNP, then there exists , then there exists LLINPINP. . Moreover, for every suchMoreover, for every such L L, there is an , there is an easier easier L’L’INPINP (i.e., (i.e., L’L’LL and and LLL’L’) )

Proof Proof (first part):(first part): For For f: f: NNNN define define LLff ::

nnLLff f(n)f(n) is is eveneven and n and nSATSAT oror

f(n)f(n) is is oddodd and n=0000… and n=0000…

2727

Ladner’s Theorem – cont.Ladner’s Theorem – cont.Let Let MM11, M, M22, M, M33 . . .. . . be all the poly-time be all the poly-time DTMDTM, ,

and and gg11, g, g22, g, g33 . . . . . . all the poly-time all the poly-time computable reductions s.t. Assume computable reductions s.t. Assume MMii(x)(x) and and ggii(x)(x) runs in time runs in time |x||x|ii. .

Define f inductively:Define f inductively:

If f(n-1) = 2jIf f(n-1) = 2j, , ff stays at the same value until stays at the same value until LLff is sure to be different from is sure to be different from MMjj at some at some sufficiently small sufficiently small xx ( (|x| < loglogn|x| < loglogn). ).

If f(n−1) = 2j+1If f(n−1) = 2j+1, , ff stays at the same value stays at the same value until it can be proved using a small until it can be proved using a small xx ( (|x| < |x| < loglognloglogn) that ) that SATSAT is not poly-time reducible is not poly-time reducible to to LLff via reduction via reduction ggjj,i.e., ,i.e., SAT(x)SAT(x)LLff(g(gjj(x)).(x)).

2828

Ladner’s Theorem – cont.Ladner’s Theorem – cont.We show that fWe show that f::

If f is stuck at 2jIf f is stuck at 2j, then (by definition of , then (by definition of LLff) ) LLff==SATSAT except finitely many inputs, and except finitely many inputs, and LLff=M=Mjj. We assumed . We assumed PPNPNP, hence, , hence, SATSATP.P.

If f is stuck at 2j+1If f is stuck at 2j+1, then (by definition of , then (by definition of LLff) ) LLffPP, and , and SAT SAT L L (using the reduction (using the reduction ggjj), ), saying again that saying again that SATSATP.P.

LLINPINP

ff is poly-time is poly-time LLffNPNPff

LL is different from any of is different from any of MMjj LLPP. . SATSAT LLff LLffNPCNPC..

2929

Backer-Gill-Solovay Theorem Backer-Gill-Solovay Theorem (1975)(1975)

Theorem (Backer-Gill-Solovay):Theorem (Backer-Gill-Solovay): There are There are oracles oracles AA and and BB such that such that

a) a) PPA A = NP= NPAA and and

b) b) PPBB NP NPBB

Note:Note: The fact that the The fact that the P=NPP=NP question question relativizesrelativizes both ways is taken as evidence both ways is taken as evidence that answering this question will be difficult, that answering this question will be difficult, because any proof technique that because any proof technique that relativizesrelativizes (i.e., is unaffected by the (i.e., is unaffected by the addition of an oracle) will not answer theaddition of an oracle) will not answer the P=NP P=NP question. question.

3030

Backer-Gill-Solovay Theorem – Backer-Gill-Solovay Theorem – cont.cont.

Proof (part a):Proof (part a):

Let Let AA be an oracle for a be an oracle for a PSPACE Complete PSPACE Complete problem. e.g., problem. e.g., TQBFTQBF - the language of - the language of true quantified Boolean formulas true quantified Boolean formulas

xx11xx22xx33xx44…F(x…F(x1’ 1’ xx2’ 2’ xx3’ 3’ xx44…)…)

NPNP and and TQBFTQBF in in NPSPACE NPSPACE NPNPAA NPSPACE NPSPACE

TQBFTQBF is is PSPACE CompletePSPACE Complete PSPACE PSPACE P PAA

NPSPACE = PSPACENPSPACE = PSPACE NPNPA A PPAA. .

It is obvious that It is obvious that PPA A NPNPAA, so in fact , so in fact NPNPA A PPAA

3131

Backer-Gill-Solovay Theorem – Backer-Gill-Solovay Theorem – cont.cont.

Proof (part b):Proof (part b):

For an oracle For an oracle BB define: define:

L(B)L(B) = The set of strings that has the = The set of strings that has the same length as some element of same length as some element of BB..

Note thatNote that L(B) L(B) NP NP (just guess the (just guess the element of element of BB of the same size) of the same size)

Let Let MM11??,, MM22

??, M, M33??...... be all the poly-time be all the poly-time

oracle- oracle- TMTMs. s.

3232

Backer-Gill-Solovay Theorem – Backer-Gill-Solovay Theorem – cont.cont.Algorithm for building a “bad” oracle B Algorithm for building a “bad” oracle B

1. Let 1. Let BB00 = = , , n = 0n = 02. 2. forfor i = 1, 2, 3,…i = 1, 2, 3,… do do

(a) (a) Choose Choose nni i > n> n s.t s.t MMiiBBi-1i-1 on input on input 00nnii asks fewer asks fewer

than than 22nnii questions questions (b)(b) ifif MMii

BBi-1i-1 accepts accepts 00nnii thenthenBBiiÃÃBBi-1 i-1

elseelseBBiiÃÃBBi-1 i-1 { { x x s.t. s.t. |x|=n|x|=nii and and xx is not is not

asked asked about by about by MMiiBBi-1i-1 }}

(c)(c) n n ÃÃ maxmaxii { length of longest query asked by { length of longest query asked by MMii

BBi-1i-1, on input , on input 00nnii }}3. 3. B B ÃÃ BBii

On input On input 00nnii , , MMiiBB runs the same as runs the same as MMii

BBi-1i-1, thus gives , thus gives wrongwrong answer for answer for 00nnii L(B)L(B) P PBB PPBB NP NPBB

3333

Polynomial Hierarchy Polynomial Hierarchy

Oracles can be used to define new classesOracles can be used to define new classes

Def:Def: CC11CC22 { L(M { L(MAA) | L(M) | L(M))CC1 1 and Aand ACC22 } }

DefDef (the class (the class i+1i+1):): 11 NP NP andand i+1i+1 NP NP ii

Or, equivalently, Or, equivalently, LL ii if there is a if there is a (i+1)-(i+1)-ary ary poly-time computable relation poly-time computable relation RR s.t. s.t.

xxLL yy11yy22yy33yy44…Q…Qiiyyii s.t. s.t.

(x, y(x, y11,, yy22,, yy33,, yy44…,y…,yii) ) RR

3434

Polynomial Hierarchy Polynomial Hierarchy Def:Def:

ii co co ii

ii P P i-1i-1

PH PH ii

Theorem:Theorem: PH PH PSPACE PSPACE

Almost all well known complexity classes Almost all well known complexity classes inside inside PSPACEPSPACE are contained in are contained in PHPH

Theorem:Theorem: If If P=NPP=NP then the hierarchy then the hierarchy collapsescollapses