CPSC 413 — Fall, 1997

Synopses of Lectures #19 – #21 (November 18 – 25, 1997)

Introduction to Complexity Theory: Encodings, P, and Reductions

1 Introduction

So far in this course, we’ve concentrated on analyzing and designing algorithms. This can be regardedas part of an attempt to prove that problems can be solved in an efficient way, and finding (and provingproperties about) efficient solutions for them.

At the end of Computer Science 313, you were introduced to quite a different activity that concernedcomputational problems, namely, proving that some of them could not be solved using realistic computers(or even idealized computers that were a bit more powerful than realistic machines, like Turing machineswith infinite work tapes) at all. As part of this, you were probably introduced to

• Turing machines, as a simple but powerful computational model;

• Church’s Thesis, which suggested that Turing machines are as powerful as several other machinemodels, and that they are powerful enough to be used to try to discover what problems can, andcannot, be solved on more realistic machines;

• an undecidable problem — the “Halting Problem” — one which could, provably, not be solved usinga Turing machine;

• reductions (or “reducibilities”) between problems, and their use to relate the “computability” of oneproblem to that of another.

For the rest of Computer Science 413, we’ll shift focus, and look at problems for which we haven’tfound efficient solutions, but that do have inefficient solutions. Our goal will generally be either to prove or(at least) provide strong evidence that these problems don’t have asymptotically efficient solutions that canbe implemented on realistic computers — or, even, on “Turing machines” (or similar machines that haveinfinite work tapes).

This may sound like a negative or pointless exercise. However, this isn’t quite true:

• It is a significant part of theoretical computer science to determine the limits of computation, and thisincludes trying to discover what computers can, and cannot, do well. We’ll be doing part of this, inthe rest of Computer Science 413.

• After you graduate, you might be put into the position of being asked to find an efficient solution fora problem that your employer wants to solve. If the problem really is “intractable,” then you won’tsucceed. Under these circumstances you’re better off (than you would be, otherwise) if you can eitherprove or at least provide strong evidence that no efficient solution for the problem exists. Ideally,the “complexity theory” that will be introduced in the rest of Computer Science 413 will help you todo that.

1

Furthermore, an understanding of this theory might help you to suggest alternatives when you’re inthis situation — possibly allowing you to help to find a related problem that can be solved efficiently,which could be used instead, or possibly allowing you to suggest a less restrictive requirement on asolution that can be met. (As an example of the latter, you might help to determine that it’s goodenough for a solution to work well on instances of bounded size.)

Much of the “algorithm analysis” material introduced at the beginning of the course will still be useful,just as it was useful when “algorithm design” was studied.

Much of the material introduced at the end of Computer Science 313 (and mentioned above) will beuseful, here, too. In particular, you should notice by the end of the term that a “complexity theory” hasbeen developed by producing a set of definitions and results that resemble the ones you saw at the endof Computer Science 313, but that this models efficiency of computation (in a way that the results about“computability” in Computer Science 313 did not).

We’ll use a machine model that was introduced in Computer Science 313 – the Turing machine. We’llidentify problems that are known or widely believed to be computationally “hard,” and we’ll see that a typeof “reduction” between problems can be used to prove that one problem is “hard” if another is — much likeyou did at the end of Computer Science 313.

Unlike “computability theory,” the “complexity theory” that we’ll see now was only developed aftercomputers were in widespread use: Most of the definitions and results that will be presented here weredeveloped in the mid- to late-1960’s and the 1970’s. Before that time, the notion of “asymptotic efficiency”was generally not perceived to be particularly important; this changed after computers were being used ona regular basis to solve larger and larger instances of problems — and it became clear that methods thata human being could use to work through a small example by hand would be inadequate when used by amachine to solve larger instances of the same problem (even though the machine was capable of workingmuch more quickly than the human being).

Unfortunately, there is another important difference between “computability theory” and “complexitytheory,” that’s been hinted at in the above introduction: While the Halting problem can be proved to beundecidable, we will only identify computational problems in these notes that are widely believed to be“intractable.” We’ll see why it is believed that they don’t have efficient solutions, and we’ll identify usefulconnections between them in these notes and in the set of notes that follow them.

Additional information on this can be found in Chapter 36 (on NP-Completeness) of Cormen, Leiserson,and Rivest’s book.

2 Model of Computation

2.1 Decision Problems

In the “algorithm design” part of this course, we’ve concentrated on “search problems” — specifically, “op-timization problems.” During the study of “complexity theory,” we’ll concentrate on “decision problems”instead.

Definition: A decision problem is a problem whose output is always either “Yes” or “No.” Thus a decisionproblem is specified by describing the inputs that make up an “instance” of the problem, and the questionwhose answer forms the output.

A Yes-instance of a decision problem is a (well formed) instance of the problem for which the corre-sponding output is “Yes,” and a No-instance of the decision problem is a (well-formed) instance for whichthe corresponding output is “No.”

2

It turns out that many search problems — especially, optimization problems — have “decision problems”that can be associated with them in a reasonably natural way. In particular, the goal of an optimizationproblem is to find some set or other structure, which is to be returned as output, that either maximizes orminimizes the value of an “optimization function” that is defined as part of the problem (or as part of aninstance of it).

A decision problem — specifically, a threshold problem — can be associated with a maximization orminimization problem by adding one more input, namely, a “threshold” k. It’s usually sufficient to expect kto be a nonnegative (or, perhaps, positive) integer, but you might occasionally want the “threshold” to assumeother real values as well (if the optimization function defined for the problem can have other values besidesnonnegative integers).

For a maximization problem, we ask whether there is a “valid” output set S for this instance of theoptimization problem, such that the value of the optimization function at this set S is greater than or equalto k — that is, this is the question about the input instance whose answer is the output for the thresholdproblem. For a minimization problem, we ask whether there is a valid output set S such that the value of theoptimization function at S is less than or equal to k.

Consider, for example, the following optimization problem (recalling that a clique in an undirected graphis a subset of the vertices, such that every pair of distinct vertices in the subset are neighbours in the graph).

MaxClique

Instance: An undirected graph G = (V,E).Output: A subset V ′ ⊆V such that V ′ is a clique in G and |V ′| ≥ |V ′′|

for every other clique V ′′ in G.

This is an example of a maximization problem: Outputs are subsets of the set of vertices of the inputgraph. Such a set is “valid” if it’s a clique, and it is “optimal” if it’s valid and as large as possible.

Since this is a maximization problem, and the “optimization function” that it defines only takes onnonnegative integers as values (it’s just the size of the set at which it’s being evaluated), the corresponding“threshold” problem would be as follows.

k-Clique

Instance: An undirected graph G = (V,E) and a positive integer k.Question: Does G have a clique of size greater than or equal to k?

You could define the same decision problem by wording the question slightly differently — asking,instead, whether a clique of maximal size in the input graph has size greater than or equal to k. The answerto the first version will be “Yes” if and only if the answer to the second is as well, so you really would bedefining the same problem, either way.

It should be clear that, if the optimization function is easy to evaluate, then the “threshold” problem iseasy to solve if the associated “optimization” problem is: Given an instance of the “threshold” problem, youcould take the part of the input that forms an instance of the optimization problem — generally, everythingexcept the “threshold” k — and solve the optimization problem, using this as an instance. Then, if S isthe correct (therefore, valid and optimal) output for this instance that is returned, you could evaluate theoptimization function at S and compare the result to k, in order to figure out whether the answer for theoriginal question (concerning the instance of the threshold problem) should be “Yes” or “No.”

Suppose, in particular, that you are given an instance of the problem k-Clique that includes an undirectedgraph G = (V,E) and a threshold k — and that the optimization problem MaxClique has an efficient solution.Then, in order to solve the k-Clique problem, you could apply your solution for the optimization problem

3

to compute a clique V ′ ⊆V in the input graph whose size is as large as possible. You should then return theanswer “Yes” if |V ′| ≥ k, and you should answer “No” otherwise.

This relationship between the optimization problem and the threshold problem is quite useful: Onceagain, it establishes that — if the optimization function can be evaluated in an efficient way — then “if theoptimization problem has an efficient solution then the decision problem has one, too.”

That probably doesn’t seem to be very interesting — but (since “¬Q ⇒ ¬P” whenever “P ⇒ Q”) itimplies that — if the optimization function can be evaluated in an efficient way — then “if the decisionproblem does not have an efficient solution, then the optimization problem does not have one, either.”

In other words: If our goal is to establish that problems do not have efficient solutions, and we areconsidering an optimization problem whose optimization function is easy to evaluate, then (in one sense)it’s “good enough” to consider the corresponding threshold problem instead — if you can manage to provethat this is intractable (has no efficient solution) then the optimization problem you started with is intractableas well.

In particular — since it’s easy to compute the size of a finite set given (a reasonable representation of)the set itself — if the problem k-Clique cannot be solved deterministically in polynomial time, then theproblem MaxClique cannot be solved deterministically in polynomial time, either.

It’s often the case that this relationship holds in the other direction as well — that is, if the thresholdproblem has an efficient solution then the optimization problem does, too. This is discussed in slightly moredetail, and an example is given, in Appendix A.

To see why this other direction might be useful, consider the following search problem and the decisionproblems defined after that. Each of these decision problems is clearly related to the search problem in someway:

Integer FactorizationInstance: An integer n≥ 2.Output: A prime p≥ 2 such that n is divisible by p.

Prime NumberInstance: An integer n≥ 2.Question: Is n a prime number?

Factor SearchInstance: An integer n≥ 2 and an integer k ≥ 1Question: Is there a prime number p such that p≥ k and n is divisible by p?

The first decision problem, Prime Number, is related to the search problem, in the sense that the answerfor the decision problem on input n is “Yes,” if and only if n is also the only correct output for the searchproblem when n is the input.

It is widely believed that the search problem Integer Factorization is (in some sense) impossible to solveasymptotically efficiently, in the “average” case as well as in the worst case — the security of at least somemodern cryptographic systems is based on this assumption.

On the other hand, there is an asymptotically efficient randomized algorithm that “almost” solves thePrime Number problem: Given an arbitrary prime integer n ≥ 2 the algorithm always returns the correctanswer, “Yes;” given an arbitrary composite number, the answer depends on the random selections that thealgorithm makes — but the probability that it gives the correct answer, “No,” is always less than one-half(and you can perform successive, independent executions of the algorithm, and decide that the answer is“No” as long as any of the answers returned by the trials is “No,” in order to increase the probability that a

4

correct answer is given). This algorithm — the Miller-Rabin primality test — is described in Chapter 33 ofCormen, Leiserson, and Rivest’s book.

Furthermore, if a plausible mathematical conjecture (the “Extended Riemann Hypothesis”) is true, thenthere is an asymptotically efficient deterministic solution for the Prime Number problem, as well.

So, this is (probably) an example such that the optimization problem is intractable but a related decisionproblem probably does have an efficient solution — so that you wouldn’t have much of a chance of provingthat the optimization problem is hard by proving that the decision problem is hard, first.

The second decision problem, Factor Search, is more closely related to the search problem — indeed,it has an asymptotically efficient solution if and only if the search problem does. To see this, suppose firstthat the search problem has an asymptotically efficient solution. Then you could solve an instance of thedecision problem (with inputs n and k) by using the algorithm for the search problem, repeatedly, in order todiscover all the prime factors of n, and you could compare each of these to k in order to solve the decisionproblem. To prove the converse, note that if the decision problem had an asymptotically efficient solution,then you could implement a “binary search,” using this solution, to compute the largest prime factor of n(and use this as a correct output for the search problem).

2.2 Encodings and Languages

2.2.1 Motivation

Since you’ve already taken Computer Science 313, you’ve already seen evidence that the job, of proving thata computational problem is “difficult,” is simplified if you select and consistently use a standard “machinemodel” — and if you choose a machine model that is as simple as possible (so that it is easier to prove thingsabout it).

As long as you don’t go too far — by choosing a machine model that is so limited that it becomesimpossible to solve problems efficiently, using the model you’ve defined, when these problems really dohave (efficient) solutions on “realistic” machines — you can still prove the results you want to using yourmachine model, and then use these results to draw conclusions about whether it is possible to perform thesecomputations, efficiently, in a more realistic setting.

However, once you’ve chosen a machine model, you must make sure that the problems you’re interestedin are ones that could be solved — at least, in principle — using the machines you’ve chosen: The inputsmust be things that these machines can “read,” and the outputs must be things that these machines can“write.”

We’ve chosen Turing machines as our machine model. Therefore, we must express all the decisionproblems we wish to solve as problems such that the instances are all strings over some finite alphabet(since these are the only things these machines can read). There are at least two ways in which Turingmachines could generate an output, “Yes,” or “No:” by halting in one (or another) designated “accepting”or “rejecting” state, or by writing the symbol “1” or “0” as an output on one its tapes. It won’t matter whichof these is used.

We must also express any “functions” that we wish to compute, as functions mapping strings over somefinite alphabet to (other) strings over some (possibly, other) finite alphabet — again, because these are theonly functions that Turing machines can compute.

2.2.2 Abstract Problems, Encodings, and Concrete Problems

However, up until now, we’ve defined both optimization and decision problems in a reasonably “abstractway,” by describing inputs by giving them names and data types — sometimes, reasonably complex ones,like undirected graphs. Outputs have been defined in the same “abstract” way, and necessary relationships

5

between inputs and outputs have been described at a fairly high (or “abstract”) level, as well. We’ll call acomputational problem an abstract problem if it has (only) been described in this way.

In order to develop a useful but simple complexity theory, that uses Turing machines as the underlyingmachine model, we must shift attention to computations on strings over some alphabet Σ (which will bediscussed in more detail later).

Suppose, now that some abstract decision problem has a set of (well formed) instances, I.

Definition: An encoding of this decision problem is a function e : I→ Σ∗, mapping instances of the problemto strings in Σ∗, which is also an injective function — that is, if i1 and i2 are two different instances of theproblem then e(i1) 6= e(i2) in Σ∗, as well.

Definition: A concrete problem is a decision problem whose set of (well-formed) instances is a subset ofthe set of strings in Σ∗.

The concrete problem corresponding to an abstract problem and encoding e is a concrete problemwhose (well-formed) instances are the encodings e(i) of all the instances i of the abstract problem, such thatthe Yes-instances of the concrete problem are the encodings e(i) of all the Yes-instances i of the abstractproblem (and, therefore, such that the No-instances of the concrete problem are the encodings e(i) of all theNo-instances i of the abstract problem, as well).

As Cormen, Leiserson, and Rivest note, it doesn’t really matter (as far as the complexity theory beingdefined is concerned) which alphabet you choose as Σ, as long as Σ contains at least two distinct symbols.Indeed, Cormen, Leiserson and Rivest use the alphabet Σ = {0,1}.

We’ll take a slightly different approach by choosing a “richer” alphabet Σ that includes the digits from0 to 9, letters of the alphabet, and sufficiently many additional symbols so that “encodings” of a variety of“abstract” decision problems are easier to develop and understand (because some complex data types can berepresented more directly). The alphabet that we’ll use for examples and tests is given in Appendix B. Bythe way, please don‘t waste time committing this to memory (or doing anything similar to that) — it’ll begiven to you again, whenever it’s needed.

To see that the choice of alphabet doesn’t matter all that much, note that the alphabet Σ given in theappendix has more than 64 = 26 symbols, but fewer than 128 = 27, so that there is a mapping ρ : Σ→{0,1}∗that maps every symbol in Σ into a different string in {0,1}∗ of length seven. This can be extended to obtaina function ρ : Σ∗ → {0,1}∗, mapping every string of length k in Σ∗ into a different string of length 7k in{0,1}∗ (and the extension should be obvious).

Now, if I is the set of instances of a concrete problem and e : I → Σ∗ is an encoding of these instancesthen the composition ρ ◦ e : I →{0,1}∗ is an “encoding” of the instances, using the alphabet {0,1} chosenby Cormen, Leiserson, and Rivest. Furthermore, the function ρ can be computed very efficiently (in lineartime); strings in {0,1}∗ that are encodings of strings in Σ∗ are easy to identify; and, finally, for any stringx ∈ {0,1}∗ such that x ∈ ρ(Σ∗), it is easy to compute a string y ∈ Σ∗ such that x = ρ(y) from the input x.These facts imply that the results about decision problems that you’ll be able to prove, using the definitionsand the alphabet Σ described in these notes, will generally imply the corresponding results that would beobtained if the alphabet {0,1} was used instead.

Having settled on an alphabet, it will also be useful to choose some “standard” ways to encode variousdata types; a short list of “standard encodings” for data types is given in Appendix C. If you’ve been given an“abstract” decision problem, and the input for this is one of the data types which has a “standard encoding”listed in the appendix, then you may use this as the “encoding” that you need, in order to define a “concreteproblem” from the abstract one. If there isn’t a “standard encoding” listed that you need, then you may needto choose one on your own.

Here are some important things to keep in mind about “standard” encodings, or whichever encodingsyou choose: You should try to choose encodings that allow fundamental operations on data to be performed

6

in a reasonable way, and you should make sure that your encodings are not unreasonably “padded:” Ingeneral, don’t choose an encoding scheme that requires instances to be represented using strings that aresignificantly longer than they need to be!

For example, it would not be a good idea to use “unary” representations of integers (such that a positiveinteger n is represented by a string of n ones) as standard encodings for positive integers, because a “unary”representation of an integer n has a length that is exponential in the length of either a binary or a decimalrepresentation of the same integer.

On the other hand, the binary, decimal, or even octal or hexadecimal representations of the same inte-ger n have lengths that only differ by (constant) factors — each length is linear in any of the others. Sinceaddition, subtraction, multiplication, division with remainder, and various comparisons of integers can allbe performed efficiently (deterministically, in polynomial time) if decimal or binary (or, octal or hexadeci-mal) representations are used to represent the integers, these encodings could all be acceptable as “standard”representations of nonnegative integers, while unary representations generally will not be — again, becausethey’re exponentially longer than any of the others. (As mentioned in the appendix, “decimal representa-tions” will be chosen as standard encodings for positive integers.)

Another encoding scheme for positive integers that one might imagine, would be one in which everyprime number p was encoded by its decimal representation “e(p)” (a string in Σ∗, since Σ includes the digitsfrom 0 to 9) and every composite number was represented by a string of the form

(e(p1),e(p2), . . . ,e(pk))

where p1, p2, . . . , pk are prime, p1 ≥ p2 ≥ ··· ≥ pk, and n = p1 · p2 . . . pk. Since the parentheses “(” and “)”and the comma “,” belong to Σ, this would be a string in Σ∗ as well.

These encodings would also be reasonably short — the length of an encoding of a positive integer nwould be linear in the length of a “standard” encoding of n (by its decimal representation).

However, this encoding would be unacceptable if we considered integer addition to be one of the “fun-damental operations” that the encoding should “support” — because we’d likely need to know how to factorintegers in order to take encodings e(n1) and e(n2) of two integers n1,n2 ≥ 2 as input, and return an encodinge(n1 + n2) of their sum as output, deterministically and in polynomial time — and, as far as we know rightnow, “factoring is hard.”

Actually, the above statement is somewhat misleading, in the sense that I haven’t proved — yet —that you would need to be able to factor integers deterministically, in polynomial time (when they’re givenby their decimal representations), in order to be able to perform the above addition deterministically, inpolynomial time. If you’d like a challenge, try to prove that there must be a deterministic polynomial-timeinteger factorization algorithm, if the above version of integer addition has such a solution.

This choice of an encoding wasn’t particularly important for Computer Science 313, because you weregenerally only interested in “computability” or “decidability.” As these examples should suggest, it’s muchmore important for Computer Science 413, because we’re interested in “tractability” — whether efficientsolutions exist for problems — as well.

Sometimes, there will be more than one encoding scheme that seems to be a reasonable choice. If this isthe case then it won’t matter much which encoding scheme you choose as long as the two encoding schemesare polynomially related, as follows:

Definition: Suppose a decision problem has a set of (well formed) instances I and suppose that e1 : I → Σ∗and e2 : I → Σ∗. These encodings are polynomially related if there are two functions f1,2, f2,1 : Σ∗→ Σ∗ thatare both computable deterministically in polynomial time (that is, in time that is bounded by a polynomialfunction of the input size, in the worst case) such that f1,2(e1(i)) = e2(i) and f2,1(e2(i)) = e1(i) for everyinstance i ∈ I.

7

You can form two different “concrete problems” from the abstract problem, depending on which of theabove two encoding schemes you choose. However, if the two encoding schemes are polynomially related(as above) then one of the concrete problems can be solved deterministically in polynomial time if and onlyif the other can as well.

Exercise: Prove that the above claim (about the two concrete decision problems) is true.

2.3 Languages

Suppose, now that a concrete decision problem has been obtained.

Definition: The language of Yes-instances of a concrete decision problem is the set of strings in Σ that areYes-instances of this problem. The language of No-instances of this problem is the set of strings in Σ thatare No-instances of this problem. Finally, the language of instances of this problem is the set of strings inΣ∗ that are (“well-formed”) instances of this problem.

Now, we’ll shift attention from “decision problems” to “languages.” Suppose A is a deterministic Turingmachine.

Definition: A accepts a string x ∈ Σ∗ if A terminates on input x and halts with output “Yes.”

Definition: A accepts a language L⊆ Σ∗ if L = {x ∈ Σ∗ | A accepts x}.

Definition: A rejects a string x ∈ Σ∗ if A terminates on input x and halts with output “No.”

Definition: A rejects a language L⊆ Σ∗ if L = {x ∈ Σ∗ | A rejects x}.

Definition: If L⊆ Σ∗, then the complement of L is the language L = {x ∈ Σ∗ | x 6∈ L}.

Definition: A decides a language L⊆ Σ∗ if A accepts L and A rejects L.

Note that this is not the same as “accepting” a language: A Turing machine that accepts a languagemight not “decide” it, because it might fail to halt on some strings that don’t belong to the language.

Definition: A decides a language L in polynomial time if A decides L and there exists some polynomial psuch that the number of steps that A takes on input x before halting is at most p(|x|), for every string x ∈ Σ∗(and, where |x| represents the length of this string).

2.4 The Complexity Class P

We’re now ready to define our first “complexity class,” P, as a set of languages that are subsets of Σ∗:

Definition: P is the set of languages L⊆ Σ∗ such that A decides L in polynomial time, for some deterministicTuring machine A.

Of course, we want to talk about decision problems, and not just about languages. We’ll say that aconcrete decision problem is “in P” if the language of Yes-instances for that decision problem is in P,according to the above definition.

We’ll say that an abstract decision problem is “in P” if the concrete decision problem, that is obtainedfrom the abstract decision problem and a standard encoding for the instances of the problem, is “in P,” asdefined above.

Here’s one thing to note about the above definitions: They don’t imply that (strings encoding) No-instances can be distinguished from strings that fail to encode instances at all.

8

2.5 How to Prove Membership of a Decision Problem in P

One way to prove that an abstract decision problem is in P is to prove that the language of Yes-instancesfor the concrete decision problem, obtained from the abstract one and a standard encoding for instances, isdecided by some deterministic Turing machine in polynomial time. That is, one way to do this is to applythe above definitions directly.

This probably doesn’t sound very helpful; I would suggest that you do in several steps.

1. First, think of a way to solve the “abstract” decision problem in a way that seems efficient. Thatis, describe an abstract function fA : I → {Yes,No} from the set I of instances of the given problemto {Yes,No}, such that fA(i) = Yes for every Yes-instance i ∈ I and such that fA(i) = No for everyNo-instance i ∈ I — and so that (you believe that) fA could be implemented in an efficient way.

2. Next, use the standard encoding and your method, and produce an algorithm for solving the “concrete”decision problem instead. If it helps, write this as pseudocode. You should probably spend some timeanalyzing this algorithm to ensure that it is correct and that it uses polynomial time (as a function ofits input size) in the worst case.

After you’ve done this, you’ll have produced an algorithm to compute a function fC : L→{Yes,No},where L⊆ Σ∗ is the language of encodings of instances of the given problem, and where fC(x) = Yes,for x ∈ L, if and only if x encodes a Yes-instance — so that, if e : I → Σ∗ is the encoding used toproduce a concrete problem from the abstract one, then fC(e(i)) = fA(i) for every instance i ∈ I.

3. In order to ensure that strings in Σ∗ that don’t encode instances are properly handled, you should provethat L ∈ P, where L is the language of instances of the given problem (as described above). That is,you should prove that there is a deterministic Turing machine that receives strings x ∈ Σ∗ as input,accepting all those strings x ∈ L, rejecting all the strings that are not in L, and such that the machineuses a number of steps on input x that is bounded by a polynomial function of |x|.It may seem discouraging to have to prove that one language (the language of instances) is in P asa step along the way to proving that a different language (the language of Yes-instances) is in P.Fortunately, it’s often the case that, if an encoding scheme was well chosen, then it’s quite easy toshow that the language of instances, L, is decided by a deterministic Turing machine in polynomialtime.

4. If you’ve got this far then you can write down pseudocode for an algorithm that decides the languageof Yes-instances for your decision problem. It does the following, on input x ∈ Σ∗:

if x ∈ L thenif fC(x) = Yes then

accept xelse

reject xend if

elsereject x

end if

A deterministic Turing machine that decides the language L could be used to implement the outer looptest, and a deterministic Turing machine that computes the function fC could be used to implementthe inner one.

9

5. Now — as much as necessary — take things down to the “Turing machine” level: That is, give as muchinformation as seems necessary to establish that the above algorithm could be implemented using adeterministic Turing machine that “decides the language in polynomial time,” as defined above. To dothis, it will be sufficient to give more information about the deterministic Turing machines that decidethe language L and that compute the function fC that are mentioned above.

It’ll generally be easier if you (at least) allow your Turing machine to have a read-only input tape, awrite-only output tape, and one or more work tapes. You can use as many work tapes as you need —as long as this number of work tapes is fixed (rather than depending on the input)!

Look for “hints” and “assumptions you’re allowed to make without proof” that concern what deter-ministic Turing machines can (and cannot) do efficiently, if you’re asked to do this.

2.6 Expectations for CPSC 413

You may be asked to prove that a decision problem is “in P” using the method that has been sketched above.Hints will be provided if an efficient algorithm doesn’t seem to be obvious. Furthermore (as suggestedabove) you can expect to receive a list of things you may assume about what deterministic Turing machinescan do efficiently, in order to keep the fussing about Turing machines to a minimum. At least, this will betrue on any test, unless the required Turing machine is so simple that this doesn’t seem to be necessary.(For example, you should certainly know enough about Turing machines to realize that there are Turingmachines that compute the identity function deterministically using polynomial time in the size of the inputin the worst case, and Turing machines that “decide Σ∗ in polynomial time,” without requiring hints orassumptions.)

You will definitely be expected to know the definitions that have been given above, and to know “how”one would go about proving that a decision problem is in P, as has been described here.

3 Reductions Between Problems: Polynomial-Time Many One Reducibility

3.1 Definitions and Key Results

If the kind of reduction introduced at the end of Computer Science 313 was a “many-one reduction” (asit frequently is), and you remember that part of the course, then you should find the next definition to befamiliar, but not identical to what you’ve seen before: It takes “efficiency” as well as “computability” intoconsideration.

Definition: If L1,L2 ⊆ Σ∗ then L1 is polynomial-time many-one reducible to L2,

L1≤mP L2,

if there exists a function f : Σ∗→ Σ∗ such that

1. for all x ∈ Σ∗, x ∈ L1 if and only if f (x) ∈ L2, and

2. f is computed by some deterministic Turing machine using time that is bounded by a polynomialfunction of its input size.

It will be useful — for several reasons — to prove that this kind of reduction holds between the languagesof Yes-instances for decision problems. We’ll occasionally “abuse notation” by saying that one of thedecision problems is “polynomial-time many-one reducible” to the other, if their languages of Yes-instancesare related in this way.

10

The next result shows one reason why these reductions are useful.

Theorem: If L1,L2 ⊆ Σ∗ and if L1≤mP L2, then if L2 ∈ P then L1 ∈ P as well.

Proof: Suppose L1≤mP L2 and L2 ∈ P.

Since L1≤mP L2 there exists a function f : Σ∗→Σ∗ such that f (x)∈ L2 if and only if x∈L1 for every string

x∈ Σ∗. Furthermore, there is a deterministic Turing machine A1 such that A1 computes f in polynomial time.That is, there is a polynomial p1 such that, for all x ∈ Σ∗, if A1 is run on input x then it terminates after atmost p1(|x|) steps, and returns f (x) as output on termination.

Since this Turing machine can only write one symbol (as output) every time it takes a step, the length ofits output string (| f (x)|) on input x cannot possibly be more than |x|+ p1(|x|) — and this is only possible ifthe machine is somehow able to “extend” its input string in order to obtain its output string.

Since L2 ∈ P, there exists another deterministic Turing machine A2 that decides L2 in polynomial time.That is, there is a polynomial p2 such that, for all y∈ Σ∗, A2 terminates on input y after taking at most p2(|y|)steps, A2 accepts y if y ∈ L2, and A2 rejects y if y 6∈ L2.

To simplify the rest of this proof we’ll require that p1 and p2 are both nondecreasing functions as wellas polynomials. We could achieve this by making these functions a bit larger than they’d otherwise need tobe (namely, by “discarding” any terms of the polynomials with negative coefficients), and we’d still havevalid upper bounds on the running times for the Turing machines when we did this.

Now consider another deterministic Turing machine, A, which does the following on input x ∈ Σ∗:

1. A simulates machine A1 on input x, computing the string f (x) after simulating at most p1(|x|) of A1’ssteps.

2. Using the result of the first step, A simulates A2 on input f (x), accepting x if A2 would acceptf (x), and rejecting x if A2 would reject f (x). Since | f (x)| ≤ |x|+ p1(|x|) and p2 is nondecreasing,p2(| f (x)|)≤ p2(|x|+ p1(|x|)), this is a polynomial function of |x| (since the sum of two polynomialsis a polynomial, and the composition of two polynomials is a polynomial as well), and this is also abound on the number of steps of A2 that are simulated in this second step.

Now, as described above, A accepts x if and only if A2 accepts f (x) — and A rejects x otherwise. By thechoice of A2, this implies that A accepts x if and only if f (x) ∈ L2, and A rejects otherwise. Furthermore, bythe choice of f , f (x) ∈ L2 if and only if x ∈ L1 — so A accepts x if x∈ L1 and A rejects x otherwise. That is,A decides L1.

Furthermore the number of steps of A1 and A2 that A simulates is bounded by a polynomial functionof the length of x (namely, p1(|x|) + p2(|x|+ p1(|x|))). Since A1 and A2 are both deterministic, A can bedesigned to perform the above simulation using at most a constant number of its own steps per step ofanother machine to be simulation (provided, at least, that A has sufficiently many work tapes). Therefore, Aperforms the above computation using polynomial time (in the length of its input).

That is, A is a deterministic Turing machine that decides L1 in polynomial time.It follows that L1 ∈ P, as claimed.

Corollary: If L1,L2 ⊆ Σ∗ and if L1≤mP L2, then if L1 6∈ P then L2 6∈ P as well.

This follows directly from the fact that “if P⇒Q then ¬Q⇒¬P.”

Another property of these reductions that will be useful later is transitivity:

Theorem: If L1,L2,L3 ⊆ Σ∗ such that L1≤mP L2 and L2≤m

P L3, then L1≤mP L3 as well.

The proof of this is left as an exercise.

11

3.2 Another Way to Prove Membership in P

Note that we now have another way to prove that a language L1 ⊆ Σ∗ is in P: Prove that L1≤mP L2, for some

language L2 that is known to be in P already, and then apply the above theorem to conclude that L1 ∈ P aswell.

3.3 How to Describe and Prove Correctness of a Reducibility

Suppose, now, that you’re given two (abstract) decision problems Q1 and Q2, and you wish to prove that oneis polynomial-time many-one reducible to the other.

As described above this means you must show that the language of Yes-instances for a concrete prob-lem, that is obtained from Q1 and some standard encoding of its instances, is polynomial-time many-onereducible to the languages of Yes-instances for a concrete problem that is obtained from Q2 and some stan-dard encoding of its instances.

Nevertheless (in spite of the fact that you eventually need to deal with encodings and languages) it’sgenerally simplest if you start with the “abstract” decision problems, and then gradually add the detailsneeded to obtain the type of “reduction” that has been described.

Suppose, now, that the abstract decision problem Q1 has a set of (well-formed) instances I1 and that theabstract decision problem Q2 has a set of (well-formed) instances I2.

1. Transforming Instances of the Abstract Problem: Try to think of a way to convert instances in I1

to instances of I2 — that is, describe a function fA : I1 → I2 (at a reasonably “high level”).

Describe this transformation in enough detail so that you can prove that it always maps a Yes-instanceof the problem Q1 to some Yes-instance of the problem Q2, and that it never maps anything else to aYes-instance of Q2 — that is, for all i∈ I1, i is a Yes-instance of Q1 if and only if fA(i) is a Yes-instanceof Q2 — and give such a proof.

You should be thinking about efficiency here, but only in the sense that you should be reasonablycertain that you could use this to generate a mapping from Yes-instances of a concrete problem corre-sponding to Q1 to Yes-instances of a concrete problem corresponding to Q2 (if reasonable encodingswere chosen). However, you shouldn’t try to prove anything formally about efficiency until the nextsteps:

2. Selection of Encodings: Determine the encodings e1 : I1 → Σ∗ and e2 : I2 → Σ∗ that should be usedto define concrete decision problems corresponding to Q1 and Q2. These may be given to you as partof a question to be solved, you may have to find them in a list of “standard” encodings (like the onein Appendix C), or you may be required to specify the encodings yourself, based on more generalinformation about how data types included in the sets of instances can be encoded.

Suppose now that Q1 is the concrete decision problem obtained from Q1 and the encoding e1 for it:Then the language of Yes-instances of Q1 is the set of strings e1(i) ∈ Σ∗, for all i ∈ I1 such that i is aYes-instance of Q1. Similarly, the language of No-instances of Q1 is the set of strings e1(i) ∈ Σ∗ forall i ∈ I1 such that i is a No-instance of Q1. Finally, the “language of instances” of Q1 is just the set ofstrings e1(i) ∈ Σ∗ for all i ∈ I1.

Similarly, suppose Q2 is the concrete decision problem obtained from Q2 and the encoding e2, sothat the language of Yes-instances of Q2 is the set of strings e2(i) ∈ Σ∗, for all i ∈ I2 such that i is aYes-instance of Q2 — and so on.

Let L1 ⊆ Σ∗ be the language of Yes-instances of Q1 and let L2 ⊆ Σ∗ be the language of Yes-instancesof Q2, as defined above.

12

For a technical reason described in the next step, you will probably need to think about a third lan-guage as well: Let L be the language of instances of Q1, as described above.

The remaining steps will probably be much easier to complete than they would be otherwise, if youcan prove that L ∈ P.

It will also be helpful if you can some string xNo ∈ Σ∗ at this point, such that xNo 6∈ L2. Either L1 = Σ∗,in which case this whole exercise trivial (actually, either trivial or impossible), or L2 6= Σ∗, so that sucha string xNo does exist.

So, we’ll conclude this description of the step by saying that you should at least try to prove that L∈Pand that you should try to identify some string xNo that does not belong to L2.

If you have a choice in the matter (that is, the encodings weren’t dictated to you), you should considerchoosing a different encoding for the first problem instead of e1 that makes this easier, if you aren’table to prove that L ∈ P using e1, and you should consider using a different encoding for the secondproblem instead of e2 if you’re having trouble choosing an element xNo that doesn’t belong to L2.

3. Identification of Function f : In this step, a suitable function f : Σ∗→ Σ∗ will be defined.

It would be tempting just to say that

“ f should map e1(i) to e2( fA(i)) for every instance i ∈ I1 of Q1,” (1)

but unfortunately that isn’t enough: The function f mentioned in the definition of “polynomial-timemany-one reduction” must be a total function from Σ∗ to Σ∗, and the above description only specifiesa partial function f : Σ∗→ Σ∗ if L = Σ∗, for L as defined in the previous step. In other words, thiswould only be enough if every string in Σ∗ encoded some instance of Q1.

However, the partial function that corresponds to the description (1) will still be useful.

Suppose, first, that you did manage to prove that L ∈ P and that you found a string xNo such thatxNo 6∈ L2, as described at the end of the previous step. Then you could define the function f : Σ∗→ Σ∗as follows: For all x ∈ Σ∗

f (x) =

{e2( fA(i)) where i ∈ I1 such that e1(i) = x, if x ∈ L,

xNo if x 6∈ L.(2)

If you had managed to prove that i is a Yes-instance of Q1 if and only if fA(i) is a Yes-instance ofQ2, for every instance i ∈ I1 of Q1 (as specified in one of the previous steps) then you can concludeimmediately that for all x ∈ Σ∗,

x ∈ L1 ⇔ f (x) ∈ L2

if f is as above.

Exercise: Prove that this is true. You will probably find it easiest to consider three (exhaustive, andmutually exclusive cases): x is an encoding of a Yes-instance of Q1, x is an encoding of a No-instanceof Q1, and x does not encode an instance of Q1 at all.

Actually, you can conclude that the above function f satisfies the above property, whether L ∈ P ornot. However, you’ll have trouble performing the next step, if you haven’t been able to show thatL ∈ P.

13

In the other case — that is, you haven’t proved that L ∈ P or you haven’t found a string xNo thatdoesn’t belong to L2 — you may still be able to continue, by “extending” the partial function shownin (1) by some other means, in order to obtain a total function f : Σ∗→ Σ∗. You’ll then have to provethat x ∈ L1 if and only if f (x) ∈ L2 for all x ∈ Σ∗, in some way — You can’t just “conclude” that it’strue (as you could, if you were in the other case).

4. Identification of Algorithm and Proof of its Correctness: At this point, you’ve defined a totalfunction f : Σ∗→ Σ∗ and proved (or given sufficient evidence) that x ∈ L1 if and only if f (x) ∈ L2. Inorder to finish you must show that f is computed by some deterministic Turing machine in polynomialtime.

I recommend that you do this in several stages, as well.

Start by describing an algorithm that could be used to compute the function f at a reasonably high level— either using English, in a few paragraphs, and as clearly as you can, or by writing the algorithm inpseudocode.

Prove (or, if it’s reasonably, clear, claim and explain why) the algorithm is correct. That is, show thatif it is given a string x ∈ Σ∗ as input then it terminates and returns f (x) as output.

Then provide as many details about how you would implement this algorithm using a deterministicTuring machine as are necessary in order for you to be able to conclude that there is a deterministicTuring machine that implements this algorithm, and so that you could estimate the number of stepsused by this machine, accurately enough to perform the step after this one.

Note: Clearly, this is a long process — and this is not supposed to be an “automata theory” course.You should expect that (when possible, especially on a test) this part of the process will be made aseasy as possible, in the sense that you’ll be given a useful list of things that you may assume that adeterministic Turing machine can do efficiently, without proof.

5. Analysis of Algorithm: Finally, prove that your algorithm, and Turing machine implementation, isalso reasonably efficient. That is, give as many details as are necessary in order to demonstrate thatthe Turing machine you’ve sketched (to implement the algorithm) never uses a number of steps thatis more than some polynomial function of the size of its input.

I recommend that you try to make claims about the algorithm that you described at a high level —in simple English, or pseudocode — whenever you can. Discuss the Turing machine that implementsthe algorithm only when this seems necessary to prove that the Turing machine uses polynomial time.The algorithm almost certainly breaks the computation down into a sequence of reasonably small andsimple steps; by taking this approach you can probably get by with arguing that each of these stepscan be implemented deterministically in an efficient way, rather than trying to do something morecomplicated.

Once again — especially on a test in this course — look for assumptions or hints that are intended tomake this last part of the process easier.

14

3.4 Example: Cliques and Independent Sets

3.4.1 Definition of Problem

Now consider the decision problem k-Clique that has already been defined, and the decision problem k-Independent Set that is defined as follows:

k-Independent SetInstance: An undirected graph G = (V,E) and a positive integer k.Question: Does G have an independent set of size greater than or equal to k?

Recall that an independent set of a graph G = (V,E) is a subset V ′ of V such that for all u,v ∈V ′ suchthat u 6= v, u and v are not neighbours. That is, the “induced subgraph” of G on the vertices in V ′ does notinclude any edges at all.

In this example we will prove that

k-Independent Set≤mP k-Clique.

One simplifying assumption will be used here (and, this is also mentioned in the appendix on encodings):It will be assumed (or required) that every undirected graph G = (V,E) that could be included in an instanceof either one of these problems has a set of vertices V such that each element of V is a string over a smalleralphabet than Σ — namely, the alphabet that only includes the digits between 0 and 9, the (upper and lowercase) letters of the alphabet, and an underscore symbol (namely, the one shown in the appendix in which Σis defined).

This will make the process of defining and recognizing encodings of graphs much easier than it wouldbe otherwise. You may make this assumption, in Computer Science 413, whenever you’re asked to provesomething about a decision problem when some component of an instance is a set of “names” or “labels”(like the set V , here).

3.4.2 Solution

Transforming Instances of the Abstract Problem: Another “graph-theoretic” definition and result willbe useful here.

Definition: Let G = (V,E) be an undirected graph. The complement graph of G, GC, is the undirected graphGC = (V,EC) with the same set of vertices V , and such that EC includes all the unordered pairs of verticesin V that are not in E. That is, distinct vertices u,v ∈ V are neighbours in the graph GC if and only if theyare not neighbours in the graph G.

Theorem: If G = (V,E) is an undirected graph and V ′ ⊆V then V ′ is a clique in GC if and only if V ′ is anindependent set in G.

This theorem is a straightforward consequence of the definitions of “clique,” “independent set,” and“complement graph,” and its proof is left as an exercise.

Corollary: If G = (V,E) is an undirected graph and k is a positive integer, then G contains an independentset whose size is at least k if and only if GC contains a clique whose size is at least k.

This is exactly what we need — for the set of instances of the decision problem “k-Independent Set” isthe set I of ordered pairs (G,k) of undirected graphs and positive integers, and the set of instances of thedecision problem “k-Clique” is this set I, as well. (Following the outline given in the previous section, wehave I1 = I2 = I).

15

The corollary asserts that if we define the function fA : I → I to be the function such that fA((G,k)) =(GC,k) for every undirected graph G and positive integer k, then fA has the properties we want: It mapsinstances of k-Independent Set to instances of k-Clique and, for every instance i ∈ I of k-Independent Set, iis a Yes-instance of k-Independent Set if and only if fA(i) is a Yes-instance of k-Clique.

Selection of Encodings: “Standard encodings” of undirected graphs and of positive integers are bothdescribed in Appendix C.

Suppose e(G) is the encoding of an undirected graph G that is described there, and that e(k) is theencoding of a positive integer k that is described there, as well. Then an encoding of an instance of eitherk-Independent Set or k-Clique, that includes the undirected graph G and the integer k, can be encoded bya string that starts with a left parenthesis, continues with the encoding e(G) of the graph, continues with acomma, followed by the encoding e(k) of the integer, and that ends with a right parenthesis:

(e(G),e(k))

It is mentioned in that appendix that you can insist that encodings of elements of sets (like V , or E) are listedin sorted order in order to make these encodings unique. Sometimes this makes things easier and somethingit makes things harder. It turns out that it makes things easier for this example, so we will assume that thisis true, for encodings of the vertex and edge sets V and E that are included in the encoding for the graph G.

Now, let

L1 = {e(i) | i is a Yes-instance of k-Independent Set},

L2 = {e(i) | i is a Yes-instance of k-Clique},

and let

L = {e(i) | i is an instance of k-Independent Set}.

As recommended in the outline for describing and proving correctness of reductions, we will continueby trying to show that L ∈ P.

Claim: L ∈ P.

Proof of Claim: This will be given in more detail than you’ll be expected to provide (or, that will begiven later in these notes), just so that you have at least one example of this kind of argument.

This will be proved by arguing that L can be decided in polynomial time by some deterministic Turingmachine. For convenience it will be assumed that the machine has quite a few work tapes (but, only aconstant number of them).

As previously described, an element of L is a string that begins with a left parenthesis, continues withan encoding of an undirected graph, a comma, an encoding of a positive integer, and ends with a rightparenthesis.

Furthermore, a standard encoding of an undirected graph has been described as a string that begins witha left parenthesis, continues with an encoding of a finite set (starting with “{”, ending with “}”, and notincluding either one of these symbols anywhere in between), a comma, an encoding of another set (with thesymbols { and } used in the same way), and that ends with a right parenthesis.

An encoding of a positive integer is its decimal representation — a nonempty string of digits that doesnot begin with “0”.

Now, using these facts — and using states of the Turing machine to keep track of the number of leftand right parentheses, left and right curly braces, and “significant” commas that you’ve seen, you can make

16

a single pass over the input string, from left to right, copying the part of the string that “should be” anencoding of an undirected graph G onto one work tape (with a single blank in front, so that you can detectthe beginning of this string later without falling off the edge of the tape), and copying the part of the stringthat “should be” the encoding of a threshold integer k onto another work tape, as you go.

Furthermore, you can confirm that the second string really is an encoding of a positive integer (or, rejectthe entire string) as you go: After all, all you need to do is check that this is a nonempty sequence of digitsthat starts with a digit that is different from “0.”

Move the tape heads back to the beginning of the tapes once you’ve done this; you’ve made only twopasses over tapes so far, so it should be clear that the number of steps used by the Turing machine up to thispoint is only linear in the size of its input.

Next — using the braces { and } to recognize the beginnings and ends of sets when you need to —make two more copies of the encoding of the vertex set V , each on another work tape, and make a copy ofthe encoding of the edge set E on yet another work tape. Once again, you’ll only need to make one or twosweeps over the strings that have been stored on work tapes in order to do this, so this will require only alinear number of steps, as well.Note: You may have noticed that at least part of this could have been combined with the first stage of theTuring machine computation, allowing you to (slightly) reduce the number of passes needed over tapes, andtherefore probably reducing the number of steps used. However, the resulting machine would have been abit more complicated than the one I am describing here. Since the one I’m describing has clearly used apolynomial number of steps (indeed, a linear number of steps) so far, nothing important would have beengained by making this change. In other words: Don’t try to optimize the algorithm or Turing machine you’redescribing at this point in a proof, at the cost of simplicity, unless it seems clear that you would need to dothis to ensure that the computation could be performed deterministically in polynomial time.

Next, we’ll validate the substring that represents an encoding of the vertex set V (or we’ll reject the inputstring). We’ve made two copies of this on two different work tapes, and we’ve “required” that the vertices inthe set be listed in lexicographical (dictionary) order. So, to ensure this, move the tape head for the secondcopy of the vertex set past the encoding of the first vertex (and comma), so that it points to the encoding ofthe second vertex — keeping the tape head for the first copy at the beginning of the encoding of the firstone. Use a constant number of passes over the encodings of these two vertices (on the different tapes) toconfirm that the name of the first vertex is “lexicographically smaller” than the name of the second vertex,as it should be, or reject the string. Move the two tape heads forward so that they point to the second andthird encodings of vertices, after that. Iterate this, comparing the encodings of “adjacent” vertices in theencoding as you go, until you’ve checked all of them. After this, you’ll either have validated the encodingof the vertex set, or rejected the input.

Since the length of the string that encodes the vertex set is linear in the sum of the lengths of theencodings of the vertices themselves, it can be shown that the number of steps used by the Turing machineto perform this part of the computation is linear in the input size, as well.

Now, move the tape heads back to the beginnings of the tapes, again. Use the two copies of the vertex set,and the copy of (what should be) an encoding of the edge set, to either validate the encoding of the edge set(and then accept the string), or reject. Taking advantage of the fact that the edges were supposed to be listedin lexicographical order as well, you should be able to convince yourself that this part of the computation canbe performed using a number of steps that is at most quadratic in the length of the input string. An exercise(for the interested, or ambitious) would be to sketch this final part of the Turing machine computation inmore detail. However, we’ll stop at this point and conclude that the claim has been satisfactorily proved.

However, here’s a “bonus:” Since the edges are supposed to be listed in lexicographical order, you canbe keeping track of the unordered pairs that are not edges in G as you perform the final part of the validation— and that you discover these in the order needed so that you can be constructing an encoding of the edge

17

set for the complement graph, on yet another work tape, as you go — without a significant increase in thenumber of steps used.

Of course, once you’ve done this (provided that the input string is accepted and encodes a graph G andinteger k), you can write an encoding of an instance that includes the complement graph GC and the sameinteger k onto an “output tape,” using a number of additional steps that is at most linear in the length of theinput string.

The final part of this step is to choose a string that doesn’t belong to L2; the “empty” string (of lengthzero) is such as string, so we’ll set xNo to be this string.

Identification of Function f : Since we’ve managed to prove that L ∈ P and to find a string xNo, this partis easy: For all x ∈ Σ∗,

f (x) =

{e((GC,k)) if x = e((G,k)) for some undirected graph G and positive integer k,

xNo otherwise,

where xNo is the empty string (as above).

Identification of Algorithm and Proof of its Correctness: This step has, pretty much, been “done al-ready,” in the sense that it’s been established that there is a deterministic Turing machine that computesthe encoding of a complement graph from the graph itself, deterministically in polynomial time, and so on.However, for the sake of completeness, let’s note that an algorithm for mapping the abstract instances would

1. Check that the input was (an encoding of) an ordered pair, including an undirected graph G = (V,E)and a positive integer k — returning the empty string as output, if this was not the case;

2. Form and return an encoding of another ordered pair, consisting of the complement graph GC =(V,EC) as defined above, and the same integer k, otherwise.

Following this high level description, you could examine each step in more detail, describing the op-erations performed by a deterministic Turing machine in sufficient detail so that it could be seen that themachine was correct (makes the decisions and computes the values that have been described) and so thatthis machine can be analyzed. Of course, this has been done already: See the details of the above proof that“L ∈ P,” and the comments that immediately follow it.

Analysis of Algorithm: This part has been done already, too. Once again, see the proof that L ∈ P andthe comments that follow it. Note that this proof gives, essentially, an analysis for each of the “high level”steps that have been described, one after the other.

Final Note: If a question like this had appeared on a test then it’s to be expected that you would havebeen told that you could assume that encodings of undirected graphs can be recognized deterministically inpolynomial time, and that the encodings of complement graphs can be computed deterministically ahead oftime — either at the beginning of the test, or ahead of time.

You would probably have been give several other “assumptions you could make” ahead of time, in orderto avoid having the assumptions themselves give too much information about the problem away.

3.5 Expectations for CPSC 413

It is expected that you will know the above definitions and that you will know (at least, in principle) howto describe polynomial-time many-one reductions between decision problems, and to prove that they exist.

18

You should be able to give a reduction between two decision problems and prove that it is correct, providedthat sufficient time is available for this, hints are supplied (where needed) in order to suggest a suitable“abstract” function fA, and so on. Information will be supplied about what Turing machines can, and cannot,do efficiently, in order to try to minimize the amount of fussing over Turing machines that is required foryou to do this.

You should also understand the applications (or implications) of these reductions — how to use them toprove that decision problems are in P, and also how to use them to prove that decision problems are not inP, by using them to relate decision problems to other decision problems.

When you do this, make sure to avoid the (unfortunately, common) mistake of giving a reduction in thewrong direction!

A Decision Problems and Search Problems

In Subsection 2.1 it was noted that one can generally define a “threshold” problem that corresponds to agiven optimization problem, in a natural way. It was noted in that section, as well, that if the optimizationproblem has an efficient solution, and the optimization function that it defines can be evaluated in an efficientway, then the threshold problem has an efficient solution, as well.

If we’re interested in proving that the optimization problem is intractable — that is, no efficient solutionexists — then this is the direction we need: If we can manage to show that the decision problem (specif-ically, the threshold problem) is NP-hard, then it follows that this problem does not have a deterministicpolynomial-time solution unless P = NP. Then, provided that the optimization function can be evaluatedefficiently, this implies that the optimization problem does not have a deterministic polynomial-time solutionunless P = NP, either (Think about it, if this isn’t clear!).

It was also noted in that subsection that the converse relationship frequently holds as well, and this isdescribed in more detail here.

Suppose, in particular, that the following conditions hold:

1. There is a way to define, recognize, and solve “base” instances of the optimization function (includingall instances with “size” zero).

2. The optimization function used to define the optimization problem is a nonnegative integer-valuedfunction, and there is an upper bound on the value that the optimization function can take, suchthat the size of a binary (or decimal) representation of this upper bound is bounded by a polynomialfunction of the size of the input instance for the problem.

If this condition is satisfied then you should be above to determine the value of the optimizationfunction that a “correct output” achieves, by using binary search (and, solving an instance of thethreshold problem whenever you want to determine whether a current guess at this value is too high,too low, or correct).

3. As usual, the output for the problem is required to be a subset of some set U . Furthermore, given anyelement x ∈U , one can form an instance of the threshold problem whose answer is “Yes” if and onlyif x is included in some correct output, for the instance of the optimization problem that you’ve startedwith.

4. Given an element x ∈ U that can be included in the output set, you can define a smaller instance Iof the optimization problem, such that a correct output for the original instance can be computedefficiently using x and any correct output for I.

19

5. Similarly, given an element x ∈U that cannot be included in the output set, you can define a smallerinstance I of the optimization problem, such that a correct output for the original instance can becomputed efficiently using x and any correct output for I.

Then, if the threshold problem has an efficient solution then the optimization problem has an efficient solu-tion as well. Rather than being proved formally, this will be “illustrated” in the following example.

A.1 Example: Finding a Maximal Clique

Consider again, the optimization problem MaxClique whose input is an undirected graph G = (V,E) andwhose output is a clique V ′ ⊆V of maximal size, and the corresponding “threshold” problem, k-Clique, thatwould accept both an undirected graph G = (V,E), and an integer k, as input, and would ask the question,“Does G contain a clique of size (at least) k?”

This pair of optimization and threshold problems satisfies the conditions that have been given above:

1. Once can define “base” instances of the optimization problem to be those whose input G = (V,E)includes no vertices at all, so that V = /0. These are easy to recognize, assuming some “reasonable”representation of an undirected graph is being used, and they are also easy to solve — the empty set /0is a correct output for any such instance of the optimization problem.

2. There is an upper bound on the size of a maximal clique in an undirected graph G = (V,E) that is evensmaller than the above condition requires: A maximal clique can include at most n vertices, where nis the size of V . Any “reasonable” encoding of an input graph G = (V,E) will have size at least n,so this is even better than what we need: The above first condition only requires that the binary ordecimal representation of an upper bound be bounded by a polynomial function of the input size —but, in this case, the bound itself is bounded by a polynomial function of the input size, as well.

This implies that you could use either binary search or a linear search to figure out the size of amaximal clique in a given undirected graph G = (V,E), by forming instances of the threshold problemthat include the graph G and integers h such that 1 ≤ h ≤ n (and asking whether a clique of size atleast h exists, when you solve this instance of the threshold problem).

Once you’ve determined the size of a maximal clique — that is, you have determined the value of theinteger h such that G includes a clique of size h but G does not include a clique of size h+1 — thenyou can satisfy the remaining conditions that are given above, as well.

3. Suppose, now, that you’ve determined that G includes a clique of size h but not of size h+1.

In order to determine whether a given node x∈G is included in some maximal clique, it is sufficient toanswer the question, “Does the induced subgraph of G on the set of neighbours of x include a cliqueof size (at least) h−1? This is an instance of the threshold problem involving a smaller graph than theinput graph (since x is not one of its neighbours), and the answer to the question is “Yes” if and onlyif there is a maximal clique in G that includes the vertex x.

4. If x is included in a maximal clique of G, then you can find such a clique by constructing a maximalclique V in the induced subgraph of G that includes the neighbours of x and then form (and return)the clique V ∪{x}.

5. If x is not included in a maximal clique of G, then you can find such a maximal clique by constructinga maximal clique V in the induced subgraph of G that includes the vertices in V \{x} (that is, all thevertices except for x), and then returning this clique as a maximal clique in the original graph, as well.

20

function MaxClique(G)

n := |V |if n = 0 then

return /0else

k := nwhile (G does not contain a clique of size at least k) do

k := k−1end while

Choose some vertex x ∈V

Compute the induced subgraph G of G on the set of neighbours of x in G

if (G contains a clique of size at least k−1) thenreturn MaxClique(G)∪{x}

elseCompute the induced subgraph G of G on the set V \{x} (that is, on all thevertices except x)

return MaxClique(G)end if

end if

end function

Figure 1: Function MaxClique

In particular, if all of these properties are satisfied, then the procedure MaxClique shown in Figure 1takes an undirected graph G = (V,E) (that is, an instance of the optimization problem) as input, uses asolution for the threshold problem k-Clique as a subroutine, and returns a maximal clique in the input graphas output. That is, it solves the optimization problem.

Suppose that the threshold problem can be solved deterministically in time O(nc) for some constant c,when given an input graph with n vertices as an instance. Suppose, as well, that it is possible to generatean induced subgraph of such a graph (when the subset of the vertices to be used has been specified) intime O(nk) for some constant k — some such constant k really does exist (and is reasonably small) if a“reasonable” representation of graphs is being used. Then this function could be implemented to solvethe optimization problem MaxClique using time O(max(nc+1,nk+1). Thus, in this case, the optimizationproblem has an efficient solution if and only if the threshold problem does, as claimed.

B A Useful Alphabet

We will now define the finite alphabet Σ, such that concrete problems have instances that are strings in Σ∗.This will be the alphabet that includes (only) the following symbols:

• the digits 0,1,2, . . . ,9;

• the upper- and lower-case letters A, B, C, . . . , Z and a, b, c, . . . , z;

• round, square, curly, and angle brackets (“(”, “)”, “[”, “]”, “{”, “}”, “〈”, “〉”);

21

• additional punctuation: period, comma, semi-colon, and colon (“.”, “,”, “;”, “:”);

• arithmetical operators, +,−, ×, /,÷;

• relational operators, <, >,≤, ≥, =, 6=,⊂, ⊃, ⊆, ⊇;

• logical connectives ∧, ∨, ¬, ⇒, ⇔;

• an “underscore” ;

but not including a blank, “ ”. We’ll require instances of the computational problems we’re interested in tobe represented as — or “encoded by” — strings of symbols in the above alphabet. The “blank” symbol hasbeen excluded from the alphabet so that you can use this symbol to terminate or to separate these strings.

C “Reasonable” Encoding Schemes

Here are a few “standard” encoding schemes for data types that will be useful, and a suggestion of ways toconstruct reasonable encoding schemes for more.

Positive integers should be encoded by their decimal representations (not padded by leading zeroes).Negative integers should be represented by a minus sign followed by the standard encoding (decimal

representation) of their absolute values.The integer zero should be represented by the string (of length one), “0”.This all provides a standard encoding for (all of) the integers.Quite a few data types include names or labels for components — for example, the vertices in an undi-

rected graph often have names. In order to make life simple, we’ll require for the rest of this that these“names” are strings over a smaller alphabet than Σ — namely, the alphabet that only includes the digits from0 to 9, underscore, and the (upper and lower case) letters of the alphabet.

If this is the case then the “standard encoding” of such a name is just the name itself — this is a stringin Σ∗.

Now, for the rest of this appendix, suppose that “T ,” T1, and T2 represent a data types for which “standardencodings” have been defined — and such that the strings that encode elements of these only include thedigits from 0 to 9, letters of the alphabet, and the underscore symbol. (So far, T , T1, and T2 would either bethe set of integers, or a subset of these, or the set of “names” as described above).

The standard encoding of a finite set of elements of type T should be a string that starts with a left curlybrace, “{”, continues with the (encodings of the) elements of this set (arranged in whatever order you want),separated by commas, and ending with a right curly brace, “}”. In other words, the “standard encoding”of the finite set is exactly the string you’d normally write down when you were explicitly listing all themembers of that set.

This doesn’t define a “standard encoding” uniquely, because the order of elements in the set isn’t spec-ified. If you’re uncomfortable with this and wish the encoding to be “unique,” then you can arrange theelements of the set so that their encodings are listed in lexicographical order (that is, the order in whichthey’d appear in a dictionary).

Now, you know how to encode the set of vertices in any undirected graph, provided that the vertices arelabeled by either integers or “names.”

If x and y belong to T , represent an ordered pair (x,y) exactly as you’d expect: by a left parenthesis, “(”,followed by the encoding for x, a comma, the encoding for y, and a right parenthesis “)”.

Represent the “unordered” pair (x,y) in exactly the same way. Of course, (x,y) = (y,x) if these represent“unordered pairs;” you can make this encoding unique, if you wish to, by requiring that the encoding of xappears before the encoding of y if and only if the encoding of x is lexicographically first.

22

Combining the above two ideas, you should be able to figure out what the standard encoding of a “set ofordered pairs” ought to be. (Ask, if this isn’t the case!)

Now, if the vertices in a graph G = (V,E) are labeled by members of T , the standard encoding of Gwill be a string beginning with “(”, followed by the standard encoding of the set V , a comma, the standardencoding of the set of unordered pairs E, and ending with “)”.

You can think of a function f : T1 → T2 whose domain is finite (that is, that is defined on only finitelymany members of T1) as being the same thing as the finite set of ordered pairs (x, f (x)) where x ∈ T1 suchthat f (x) is defined — and then you can use (or slightly extend) the notion of a standard encoding of a finiteset, in order to obtain a “standard encoding” for this kind of finite function.

Represent a finite sequence r1,r2, . . . ,rn by a left angle bracket, “〈,” followed by the standard encodingsof r1,r2, . . . ,rn in order, and separated by commas, and ending with a right angle bracket, 〉.

A reasonable “standard encoding” of Boolean formulas that are functions of variables x1,x2, . . . ,xn andthat use the logical connectives ∧, ∨, and ¬, is described in Question #2 on Problem Set #8.

Sometimes you might want to “encode” sentences — sequences of words — strings of alphanumericcharacters (letters or digits) — that are separated by spaces. Represent the words by themselves and replacethe blanks by underscore symbols; then, you’ve got an encoding as a string in Σ∗.

And, that should be enough to get you started!

23