instructor’s manual volume i models of computation

Instructor’s Manual

Volume I

Models of ComputationExploring the Power of Computing

John E. SavageBrown University

An imprint of Addison Wesley Longman, Inc.

Reading, Massachusetts � Harlow, England � Menlo Park, CaliforniaBerkeley, California � Don Mills, Ontario � Sydney

Bonn � Amsterdam � Tokyo � Mexico City

Copyright c 1998 Addison Wesley Longman, Inc.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system,or transmitted, in any form or by any means, electronic, mechanical, phtocopying, recording,or otherwise, without the prior written permission of the publisher. Printed in the UnitedStates of America.

Introduction

This manual contains solutions to all of the problems in the first five chapters of my book.These 200 problems represent just under 50% of the 427 problems in the book.

I have prepared these solutions because I believe that the material in these chapters exceptfor the last half of Chapter 2 plus a little bit of Chapter 8 can serve as the basis for an excellentfirst course in theoretical computer science. Instructors wishing to investigate this possibilityare encouraged to consult the web site:

http://www.cs.brown.edu/people/jes/book/book.html.

John E. SavageFebruary 14, 2000

1

Contents

1 The Role of Theory in Computer Science 51.1 Solutions 5

2 Logic Circuits 152.1 Solutions 15

3 Machines with Memory 373.1 Solutions 37

4 Finite-State Machines and Pushdown Automata 594.1 Solutions 59

5 Computability 915.1 Solutions 91

3

C H A P T E R

The Role of Theory inComputer Science

1.1 SolutionsMATHEMATICAL PRELIMINARIES

1.1 Problem: Show that the sum S(k) below has value S(k) = 2k � 1:

S(k) = 2k�1 + 2k�2 + � � �+ 21 + 20

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The proof is by induction. The base case is k = 1 for which it is immediatethat S(1) = 1 = 21�1. Suppose that S(j) = 2j�1 for 1 � j � k�1. We show thatit holds for j = k. We write S(k) = 2k�1 + S(k� 1) = 2k�1 + 2k�1� 1 = 2k� 1.

SETS, LANGUAGES, INTEGERS, AND GRAPHS

1.2 Problem: Let A = fred, green, blueg, B = fgreen, violetg, and C = fred, yellow,blue, greeng. Determine the elements in (A \C)� (B �C).

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: (A\C) = fred, blueg and B�C = fvioletg. Thus, (A\C)�(B�C) =f(red, violet), (blue, violet)g.

1.3 Problem: Let the relation R � N �N be defined by pairs (a, b) such that a and b

have the same remainder on division by 3. Show that R is an equivalence relation.

5

6 Chapter 1 The Role of Theory in Computer Science

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: We need to show that this relation is reflexive, symmetric, and transitive. Itis reflexive if (a, a) 2 R which holds because a and a have the same remainder modulo3. It is symmetric because (a, b) 2 R exactly when both have the same remaindermodulo 3; that is, (b, a) 2 R also. It is transitive because if a and b have the sameremainder modulo as do b and c, then a and c have the same remainder modulo 3 and(a, c) 2 R.

1.4 Problem: Let R � A�A be an equivalence relation. Let the set E[a] be the elementsin A equivalent under the relation R to the element a. Show that for all a, b 2 A theequivalence classes E[a] and E[b] are either equal or disjoint. Also show that A is theunion of all equivalence classes.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: We prove that E[a] and E[b] are either equal or disjoint by contradiction.We suppose that it is false and show that a contradiction results.

Suppose that c 2 E[a] \E[b] but E[a] 6= E[b]. Because R is an equivalence relation,for every element u 2 E[a],uRc. Also, for every element v 2 E[b], vRc. By symmetryand transitivity, uRv. Thus, every element in E[a] is in E[b] and vice versa; that is,E[a] = E[b].

Since for each element a 2 A, aRa, it follows that every element of A is in someequivalence class.

1.5 Problem: In terms of the Kleene closure and the concatenation of sets, describe thelanguages containing the following:

a) Strings over f0, 1g beginning with 01.

b) Strings beginning with 0 that alternate between 0 and 1.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) f01gf0, 1g�b) 0(f10g�1 [ f10g�)

1.6 Problem: Describe an algorithm to convert numbers from decimal to binary notation.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The binary representation of a number is an expression of the form xn�12n�1

+ � � �+ x121 + x020. Thus, if we divide the decimal number by 2 the remainder willbe x0 and the quotient will be xn�12n�2 + � � �+ x1. To find the value of x1, repeatthe process on the quotient, etc. The computation of the quotient and remainder of adecimal number upon division by 2 is straightforward.

1.7 Problem: A graph G = (V ,E) can be described by adjacency lists, one list for eachvertex in the graph. The adjacency list for vertex v 2 V is a list of vertices to whichthere is an edge from v. Generate adjacency lists for the two graphs of Fig. .1.2.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Let Li be the list of vertices to which vertex vi has an edge. For the graphin Fig. 1.2(a) the lists are the following: L1 = fv3g, L2 = fv3g, L3 = fv5g, and

1.1 Solutions 7

L4 = fv5g. For the graph in Fig. 1.2(b) the lists are the following: L1 = fv4, v5g,L2 = fv4, v5g, L3 = fv6, v7g, L4 = fv6, v7g, L5 = fv8g, L6 = fv8g.

TASKS AS FUNCTIONS

1.8 Problem: Let Z5 be the set f0, 1, 2, 3, 4g. Let the addition operator � over this setbe modulo 5; that is, if x and y are two such integers, x� y is obtained by adding xand y as integers and taking the remainder after division by 5. For example, 2� 2 =4 mod 5 whereas 3 � 4 = 7 = 2 mod 5. Provide a table describing the functionf� : Z5 �Z5 7! Z5.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: x y f�0 0 00 1 10 2 20 3 30 4 41 0 11 1 21 2 31 3 41 4 0

x y f�2 0 22 1 32 2 42 3 02 4 13 0 33 1 43 2 03 3 13 4 2

x y f�4 0 44 1 04 2 14 3 24 4 3

1.9 Problem: Give a truth table for the Boolean function whose value is True exactly wheneither x or y or both is True and z is False.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The truth table for this function, f(x, y, z), is given below.

x y z f(x, y, z)

0 0 0 0

0 0 1 0

0 1 0 1

0 1 1 0

1 0 0 1

1 0 1 0

1 1 0 1

1 1 1 0

RATE OF GROWTH OF FUNCTIONS

1.10 Problem: For each of the fifteen unordered pairs of functions f and g below, determinewhether f(n) = O(g(n)), f(n) = (g(n)), or f(n) = �(g(n)).

a) n3; c) n6; e) n3 log2 n;

b) 2n log2 n; d) n2n; f ) 22n :


. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The following is the ordering of functions by their asymptotic size: n3 =O(n3 logn); n3 logn = O(n6); n6 = O(n2n); n2n = �(2n log2 n); 2n log2 n =O(22n).

1.11 Problem: Show that 2:7n2 + 6pndlog2 ne � 8:7n2 for n � 3.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Subtracting 2:7n2 from both sides and then dividing each side by 6 we havethe following inequality which is equivalent to the above.

dlog2 ne � n1:5

This inequality certainly holds for n = 1 and 2. We show that it holds for n � 3.Since dlog2 ne � log2 n + 1, if we can show that log2 n+ 1 � n for n � 3, then thedesired result holds because n � n1:5. Subtracting 1 from both sides of the inequalitylog2 n+ 1 � n, raising both terms to the exponent 2, and dividing them by n we havethe equivalent inequality 1 � 2n�1=n. If we can show that this holds, we are done. Butf(n) = 2n�1=n is an increasing function of n for n � 3 because f(n + 1)=f(n) =2n=(n+ 1) � 1 for n � 1. Thus, f(n+ 1) is larger than f(n) for all n � 1. Becausef(2) = 1 it follows that 1 � 2n�1=n for n � 3, and the desired result follows.

METHODS OF PROOF

1.12 Problem: Let Sr(n) =Pn

j=1 jr denote a sum of powers of integers. Use proof by

induction to show that the following identities on arithmetic series hold:

a) S2(n) =n3

3 + n2

2 + n6

b) S3(n) =n4

4 + n3

2 + n2

4

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) Note that S2(n+1) = S2(n)+(n+1)2 = S2(n)+n2+2n+1. We now expandthe expression given above for S2(n+1) and show that the two expressions are thesame.

S2(n + 1) =(n + 1)3

3+(n + 1)2

2+(n + 1)

6

=(n3 + 3n2 + 3n+ 1)

3+

(n2 + 2n+ 1)2

+(n+ 1)

6

=n3

3+n2

2+n

6+ n2 + 2n+ 1

= S2(n) + (n + 1)2 = S2(n) + n2 + 2n+ 1

b) Note that S3(n+1) = S3(n)+(n+1)3 = S2(n)+n3 +3n2 +3n+1. We nowexpand the expression given above for S3(n+1) and show that the two expressions

1.1 Solutions 9

are the same.

S3(n+ 1) =(n+ 1)4

4+(n+ 1)3

2+(n+ 1)2

4

=(n4 + 4n3 + 6n2 + 4n+ 1)

2+

(n3 + 3n2 + 3n+ 1)2

+(n2 + 2n+ 1)

4

=(n+ 1)4

4+(n+ 1)3

2+(n+ 1)2

4+ n3 + 3n2 + 3n+ 1

COMPUTATIONAL MODELS

1.13 Problem: Produce a circuit and straight-line program for the Boolean function de-scribed in Problem 1.9.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: A table is given below for the function whose value is True exactly wheneither x or y or both is True and z is False. Combining the minterms xyz and xyz weproduce the product yz. Similartly, combining xy z and xyz we produce the productxz. Thus, f has the SOPE expansion yz _ xz which simplifies to (x _ y) ^ z. Astraight-line program for this function is shown below.

x y z f(x, y, z)0 0 0 00 0 1 00 1 0 10 1 1 01 0 0 11 0 1 01 1 0 11 1 1 0

g1 := x ;g2 := y ;g3 := z ;g4 := :g3 ;g5 := g1 _ g2 ;g6 := g5 ^ g4 ;

1.14 Problem: A state diagram for a finite-state machine is a graph containing one vertex(or state) for each pattern of data that can be held in its memory and an edge fromstate p to state q if there is a value for the input data that causes the memory to changefrom p to q. Such an edge is labeled with the value of the input data that causes thetransition. Outputs are generated by a finite-state machine when it is in a state. Thevertices of its state diagram are labeled by these outputs.

Provide a state diagram for the finite-state machine described in Fig. 1.5(b).

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The state diagram is shown in Figure 1.1.

1.15 Problem: Using the straight-line program given for the full-adder circuit in Section 1.4.1,describe how such a program would be placed in the random-access memory of theRAM and how the RAM would run the fetch-and-execute cycle to compute the val-ues produced by the full-adder circuit. This is an example of circuit simulation by aprogram.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: We imagine placing statement i in the ith word of the RAM. This wordspecifies the operation to be performed as well as the numbers of the words containing


0

1

0

1

q0=0 q1=1

Initial

Figure S.1.1 The FSM associated with Problem 1.14

the arguments needed for the operation. Proceding from the lowest to the highestnumbered statement, the operand(s) of the current statement is (are) fetched and theoperation evaluated. The value of the i operation is used to replace its descriptionthat was previously stored in the ith word. This program executes O(n) steps on astraight-line program with n statements.

1.16 Problem: Describe the actions that could be taken by a Turing machine to simulate acircuit from a straight-line program for it. Illustrate your approach by applying it to thesimulation of the full-adder circuit described in Section 1.4.1.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Assume that the statements of the straight-line program are placed in succes-sive cells just as described for the RAM in the solution to Problem 1.15. However, sinceaddressing of cells is not possible, it is necessary for the TM to search from the currentstatement backward to find the values of its operands. Once it finds these operands,it replaces the current word with the statement number followed by the value of theoperation.

The backward search for operands is facilitated by representing statement numbers inunary (by a string of 1’s) and by placing markers on statement numbers. This allowsthe TM to bounce back and forth between statement numbers preceding the currentstatement to find the statement that contains the value in question.

If the program has n steps, this procedure usesO(n2) units of time for each step becauseO(n) steps are used for each of the O(n) digits in the unary representation of thecurrent statement number. Since there are n statements to be simulated, this algorithmwill run in timeO(n3).

1.17 Problem: Suppose you are told that a function is computed in four time steps by avery simple finite-state machine, one whose logic circuit (but not its memory) can berealized with four logic gates. Suppose you are also told that the same function cannotbe computed by a logic circuit with fewer than 20 logic gates. What can be said aboutthese two statements? Explain your answer.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Given a function that is computed in four time steps by finite-state machinewhose logic circuit can be realized with four logic gates, we know that it has a minimalcircuit size that is at most 16 because of the construction based on unwinding the FSMloop. If you are told that this function cannot be computed by a logic circuit with fewer

1.1 Solutions 11

than 20 logic gates, there is a contradiction between this statement and the above citedconstruction.

1.18 Problem: Describe a finite-state machine that recognizes the language consisting ofthose strings over f0, 1g that end in 1.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: This machine has two states, the initial state q1 and an accepting state q2. Inq1 on input 1, move to q2 and to q1 on input 0. In q2 on input 1, move to q2 and to q1

on input 0.

1.19 Problem: Determine the language generated by the context-free grammarG = (N ,T ,R, S) where N = fS, M, Ng, T = fa, b, c, dg andR consists of the rules given below.

a) S ! MN

b) M ! aMb

c) M ! ab

d) N ! cNd

e) N ! cd

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: It is straightforward to show by induction that all the strings over the al-phabet of terminals and non-terminals generated by this grammar are of the formarMbrcsNds for r, s � 0. Thus, the terminal strings are of the form akbkcldl fork, l � 1.

COMPUTATIONAL COMPLEXITY

1.20 Problem: Using the rules for the red pebble game, show how to pebble the FFT graphof Fig. 1.10 with five red pebbles by labeling the vertices with the time step on which itis pebbled. If a vertex has to be repebbled, it will be pebbled on two time steps.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The time of pebblings of vertices when three and five pebbles are used areshown in Figure 1.10.

1.21 Problem: Suppose that you are told that the n-point FFT graph can be pebbled withpn pebbles in n=4 time steps for n � 37. What can you say about this statement?

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: On page 25 of the book it is said that S and T must satisfy (S + 1)T �n2=24. Since S =

pn and T = n=4, it follows that the inequality must be satisfied

which is equivalent to 6(pn + 1) � n. But the left-hand side has value more than 42

whereas the right-hand side has value 37 for n = 37, a contradiction.

1.22 Problem: You have been told that the FFT graph of Fig. 1.10 cannot be pebbled withfewer than three red pebbles. Show that it can be pebbled with two red pebbles inthe red-blue pebble game by sketching how you would use blue pebbles to achieve thisobjective.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: To pebble an internal vertex v place a red pebble on each of immediatepredecessors and slide one of these pebbles to v. If the next vertex to be pebbled doesnot have vertex v as one of its predecessors, replace the red pebble on v with a bluepebble.


0

0b0

0

0

0

Step 1

0

Step 3

0 0

Step 4

0 0

0

b2

c2

a0

b1

c1

0

c0

a1

b1

c2

0

0

c1

a0

b0

c0

0

0

c2

a1

b0

c1

0

0

c0

a0

0

c2

0

0

c1

0

b1

c0

b0

0

Step 20 0 a1

0 0a1

Figure S.1.2 A systolic for Problem 1.23.

PARALLEL COMPUTATION

1.23 Problem: Using Fig. 1.12 as a guide, design a systolic array to convolve two sequencesof length two. Sketch out each step of the convolution process.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: See Fig. S.1.2.

1.24 Problem: Consider a version of the PRAM consisting of a collection of RAMs (seeFig. 1.13 also shown here as Fig. S.1.3) with small local random-access memories thatrepeat the following three-step cycle until they halt: a) they simultaneously read oneword from a common global memory, b) they execute one local instruction using localmemory, and c) they write one word to the common memory. When reading andwriting, the individual processors are allowed to read and write from the same location.If two RAMs write to the same location, they must be programmed so that they writea common value. (This is known as the concurrent-read, concurrent-write (CRCW)PRAM.) Each RAM has a unique integer associated with it and can use this number todecide where to read or write in the common memory.

Show that the CRCW PRAM can compute the AND of n Boolean variables in twocycles.Hint: Reserve one word in common memory and initialize it with 0 and assign RAMsto the appropriate memory cells.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: One memory location in global memory is reserved for each variable and oneadditional location is reserved for the value of the function. This location is initializedat 1. One PRAM processor is assigned to each location containing a variable. It reads its

1.1 Solutions 13

RAMRAM

Common Memory

p1 p2 pn

RAM

Figure S.1.3 The PRAM model is a collection of synchronous RAMs accessing a commonmemory.

variable on one cycle and writes 0 to the location containing the value of the functionif its variable is 0. Thus, all the processors that write, write a common value.

C H A P T E R

Logic Circuits


2.1 Problem: Show that the following identities on geometric series hold:

P0(s) =sX

j=0

aj =(as+1 � 1)(a � 1)

P1(s) =sXj=0

ajj =a

(a � 1)2(sas+1 � (s + 1)as + 1)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: (a) The proof is by induction. Clearly, P0(0) = 1 both in the sum and inthe putative closed form for the sum. Assume that

P0(j) =(aj+1 � 1)(a � 1)

for 0 � j � s� 1. We show that it holds for j = s as well.

P0(s) = P0(s � 1) + as = ((as � 1) + (a� 1)as) =(a� 1)

from which the desired result follows by algebraic manipulation.

15

16 Chapter 2 Logic Circuits

2:5

2:0

1:5

1:0

0:5

3:0

1 2 3 4 5 6 7 8 9 10 11 12 13

n

logn

log(n+ 1)

Figure S.2.1 The natural logarithm of the factorial n! isP

n

k=1 ln k, which is bounded belowbyRn

1ln x dx and above by

Rn

1ln(x+ 1) dx.

(b) This proof is by direct manipulation. Observe that

P1(s) + (s + 1)as+1 =s+1Xj=0

ajj =sX

k=0

ak+1(k + 1)

=sX

k=0

ak+1k +sX

k=0

ak+1 = aP1(s) + aP0(s)

The result follows by algebraic manipulation.

2.2 Problem: Derive tight upper and lower bounds on the factorial function n! = n(n �1) � � �3 2 1.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: From the caption on Fig. 2.23 (repeated as Fig. S.2.1) it follows that

Z n

1lnx dx � lnn! �

Z n

1ln(x+ 1) dx

The integralR balnx dx has the closed form solution b(ln b � 1) � a(ln a � 1). A

change of variable in the second integral shows that it is equivalent toR n+1

2 ln y dy. Itfollows that

n lnn� (n � 1) � lnn! � (n+ 1) ln(n+ 1)� (n � 1)� ln4

Exponentiating both sides we have the following bounds:

nne�(n�1) � n! � 14(n+ 1)(n+1)e�(n�1)

2.1 Solutions 17

(a) (b)

Figure S.2.2 Complete balanced binary trees a) of depth two and b) depth 4.

2.3 Problem: Let T (d) be a complete balanced binary tree of depth d. T (1), shown inFig. 2.24(a) (Fig. 2.24 is repeated here as Fig. S.2.2), has a root and two leaves. T (d) isobtained by attaching to each of the leaves of T (1) copies of T (d� 1). T (3) is shownin Fig. 2.24(b) (and in Fig. S.2.2(b)).

a) Show by induction that T (d) has 2d leaves and 2d � 1 non-leaf vertices.

b) Show that any binary tree (each vertex except leaves has two descendants) with n

leaves has n � 1 non-leaf vertices and depth at least dlog2 ne.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) Let L(d) and N (d) be the number of leaves and non-leaf vertices in T (d) respec-tively. By induction we show that L(d) = 2d and N (d) = 2d � 1, respectively.The base case has L(1) = 2 and N (1) = 1 which follows by inspection of T (1).For the inductive hypothesis we assume that L(e) = 2e and N (e) = 2e � 1 fore � d�1. Since T (d) is formed by attaching two copies of T (d�1) to the leavesof T (1), it follows that L(d) = 2L(d � 1) and N (d) = 1 + 2N (d � 1) fromwhich the result follows.

b) We show by induction that every binary tree with n leaves has n � 1 non-leafvertices. The statement is true for T (1) which has two leaves. Suppose it is truefor every binary tree with n � 1 leaves. Given a binary tree with n leaves, theremust exist some non-leaf vertex both of whose subtrees consist of leaf vertices. (Ifnot, at least one subtree of every vertex is not a leaf vertex and the graph has cycle.)Remove such a non-leaf vertex and replace it by a leaf. Since the new tree has n�1leaves, it n�2 non-leaf vertices and the original tree must have had n�1 non-leafvertices.

It is an easy proof by induction that a binary tree of depth d has at most 2d leaves.The base case applies to T (1). Let 2d�1 < n � 2d. Since a binary tree of depthd has two subtrees each of which has depth at most d � 1, a binary tree with n

leaves cannot have depth less than d�1 because such a tree has insufficiently manyleaves. Thus, it must have depth at least d where by definition d = dlog2 ne.


BINARY FUNCTIONS AND LOGIC CIRCUITS

2.4 Problem:

a) Write a procedure EXOR in a language of your choice that writes the descriptionof the straight-line program given in equation (2.2).

b) Write a program in a language of your choice that evaluates an arbitrary straight-line program given in the format of equation (2.2) in which each input value isspecified.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) This is straightforward. Just use one print statement for each line of the straight-line program.

b) If it is known in advance how many statements the program contains, then a lan-guage such as Pascal, which does not have dynamic storage allocation, can be used.Create an array that will contain the value produced at each step of the program.Assume the statements in the straight-line program are provided in an order suchthat each statement uses operands that have been produced earlier. When a newstatement arrives, use the array to find the value of its operands. Compute thevalue of the operator on these operands and store its value in the array.

If the length of the input is not known in advance, Scheme, or some other languagesuch as C which supports dynamic storage allocation must be used.

2.5 Problem: A set of Boolean functions forms a complete basis if a logic circuit can beconstructed for every Boolean function f : Bn 7! B using just functions in .

a) Show that the basis consisting of one function, the NAND gate, a gate on twoinputs realizing the NOT of the AND of its inputs, is complete.

b) Determine whether or not the basis fAND, ORg is complete.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) Since the NAND gate computes the function fnand(x, y) = x ^ y, the NOT func-tion, fnot(x) = x, can be realized from fnand(x,x). Applying this to the outputof fnand(x, y), the AND gate is realized. Finally, by DeMorgan’s rule it followsthat fnand(x, y) = x _ y from which it follows that fnand(x, y) is the functioncomputed by the OR gate. Thus, from the NAND gate we can realize each of thegates in the standard basis, which we know is complete.

b) The basis fAND, ORg is not complete because the NOT gate cannot be realized byit. To show this we show by induction that every function computed by a circuitover this basis is monotone, that is, increasing any input from 0 to 1 cannot causethe output to decrease from1 to 0. This is true for any one input gate. Considerthe last gate of the circuit and show that it must also hold in this case. But theNOT gate is non-monotone.

2.6 Problem: Show that the CNF of a Boolean function f is unique and is the negation ofthe DNF of f .

2.1 Solutions 19

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Two CNF’s that differ must differ in a maxterm; that is, one has the maxtermand the other does not. Since each maxterm in a CNF has value 0 on exactly one setof values for its variables, one CNF represents a function with value 0 on this set ofvalues whereas the other represents a function with value 1. It follows that the CNF ofa function is unique.

To show that the CNF is the negation of the DNF, observe that the former is theconjunction (AND) of maxterms (ORs of literals) while the latter is the disjunction(OR) of minterms (ANDs of literals). By DeMorgan’s rules, the negation of the OR ofterms is the AND of their complements. Also, by these rules the complement of theANDs of literals is the OR of their complements. In this case, the last terms that arecomplemented are literals whose complements are also literals and the result follows.

2.7 Problem: Show that the RSE of a Boolean function is unique.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: To show that the ring-sum expansion RSE of a Boolean function f is uniquewe suppose that f has two distinct RSEs, E1 and E2, and show that a contradictionresults. The RSE consists of the EXCLUSIVE OR of the product (AND) of variables off . Let the degree of a product be the number of variables in it. Since E1 and E2

differ, there must be a smallest degree, say d, such that one of them, say E1, contains aproduct of this degree but the other does not. It follows that both E1 and E2 containa common expression Ec consisting of the EXCLUSIVE OR of terms of degree d � 1or less. If we delete these products from E1 and E2 giving F1 and F2, we have twodifferent RSE’s for g = f �Ec. If we can show that g has a unique RSE, then the sameapplies to f .

The lowest degree products in F1 and F2 all have degree d. Let p be a product of degreed in F1 but not F2. Set all variables in p to 1 and the rest to 0. Not only does this set to0 all products of degree d except for p, it also sets to 0 all products of higher degree. Itfollows that F1 has value 1 but F2 has value 0. Thus, g cannot be represented correctlyby both F1 and F2 and we arrive at a contradiction. Thus, g (and f ) have a uniqueRSE.

2.8 Problem: Show that any SOPE (POSE) of the parity function f(n)�

has exponentiallymany terms.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: First, consider the DNF of the parity function. Each function of n variableshas a truth table with 2n rows. For each value for x1,x2, : : : ,xn�1,

f(n)�

(x1,x2, : : : ,xn�1, 0) 6= f(n)�

(x1,x2, : : : ,xn�1, 1)

It follows that for exactly half of the rows the function has value 1. Thus, the DNF ofa parity function has 2n=2 minterms.

The DNF is just a special case of a SOPE. To prove that every SOPE has exponentiallymany terms it suffices to show that no product in the SOPE has less than n variables,or what it is equivalent, that the DNF cannot be simplified.


Consider the SOPE of the parity function f (n)�

and suppose that it contains the productxc1

1 ^ xc22 ^ � � � ^ xcrr for r < n where xcii = xi if ci = 1 and xcii = xi if ci = 0.

Give values to these variables so the the product has value 1. Then the SOPE has value1. Also set the variables not in this product to the values which cause the f (n)

�to have

value 1. Now change the value of a variable not in the product. This does not cause a

change in the value of the SOPE but it does cause a change in the value of f (n)�

. Thus,

a SOPE with a product of less than n variables cannot represent f (n)�

.

The proof for the POSE is exactly the dual. f (n)�

has value 0 on exactly half the pointsin its domain. If some clause does not include all variables, it can be set to 0 withoutfixing all the variables. Thus, the POSE remains 0 as one of the remaining variables is

varied, contradicting the definition of f (n)�

.

2.9 Problem: Demonstrate that the RSE of the OR of n variables, f (n)_

, includes everyproduct term except for the constant 1.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: The function f

(n)_ (x1,x2, : : : ,xn) has value 1 on every n-tuple of weight

k � 1. We show by contradiction that every product term except the constant term isin the RSE of this function. To show that the constant term in the RSE of f (n)_ is 0,observe that all product terms are 0 when every variable is 0.Suppose xi1xi2 � � �xik is the product term containing the fewest variables that is miss-ing from the RSE. That is, every product term containing more than k variables is inthe RSE. Now set each variable except those in this product to 0. This causes everyterm containing more than k variables to have value 0. Since there is only one productterm containing these k variables, if we set them to 1 and all other variables to zero,every product term in the RSE except for xi1xi2 � � �xik is 0. However, since this termis missing from the RSE, it must have value 0, contradicting the assumption that it isthe RSE of the OR function.

2.10 Problem: Consider the Boolean function f(n)mod3 on n variables, which has value 1

when the sum of its variables is zero modulo 3 and value 0 otherwise. Show that it hasexponential-size DNF, CNF, and RSE normal forms.

Hint: Use the fact that the following sum is even:

X0�j�k

�3k3j

�. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: We show exponential-size DNF and CNF by showing that the number of 1’sand 0’s in the truth table for f (n)mod3 is exponential. The number of 1’s plus the numberof 0’s is 2n. We give upper and lower bounds on the number of 1’s from which theseresults follow.Cnk is the binomial coefficient defined below. The identity given for it holds when

1 � k � n� 1, as can be easily shown.

Cnk =

�n

k

�= Cn�1

k +Cn�1k�1

2.1 Solutions 21

From this we have the following identity and bounds when 2 � k � n� 2:

Cnk = Cn�2

k + 2Cn�2k�1 +Cn�2

k�2

Cnk � 2(Cn�2

k + Cn�2k�1 +Cn�2

k�2 )

Cnk � Cn�2

k +Cn�2k�1 +Cn�2

k�2

Let f(n) =P

bn=3cj=0 Cn

3j. Thus, the number of 1’s in the truth table for f (n)mod3 is f(n)and the number of 0’s is 2n � f(n). We show this implies that both the number of1’s and the number of 0’s grow exponentially with n and the DNF and CNF of f (n)mod3have expontial size.Let Æ(x) be the Dirac delta function which is 1 if x > 0 and 0 if x = 0. If 2 � 3j �n � 2, we have 1 � j � b(n � 2)=3c. Let a = bn=3c and b = b(n � 2)=3c. Then,a � b = 1 unless n = 3k + 2 for some integer k in which case a � b = 0. It followsthat

f(n) =�Pb

j=1

�Cn�2

3j + 2Cn�23j�1 + Cn�2

3j�2

��+ 1 + Æ(a� b)Cn

3a

� 2�Pb

j=1

�Cn�2

3j + Cn�23j�1 + Cn�2

3j�2

��+ 1 + Æ(a� b)Cn

3a

� 2n�1 +Cnn�2

Here we use the fact that Cn3a is an decreasing function of a and that 3a � n�2. Also,

because

3b+ 1 = 3b(n� 2)=3c+ 1

� 3

�n� 2

3� 2

3

�+ 1

= n� 4 + 1 = n� 3

we have the following lower bound:

f(n) � Pbj=1

�Cn�2

3j + Cn�23j�1 + Cn�2

3j�2

�+ 1

= 2n�2 �Pn�2j=3b+1 C

n�2j

� 2n�2 � n+ 2

Thus, 2n�2 � n+ 2 � f(n) � 2n�1 + n(n� 1)=2.

Consider next the RSE for f (n)mod3. The RSE can be written as the integer sum modulotwo of a constant and products of one or more variables that appear in the RSE. If weset all variables of f (n)mod3 to 0, the function has value 1. Since all products in the RSE

are zero, it follows that the constant is 1. We can show by considering cases that allproducts of one and two variables are present. Since the function is symmetric, if aproduct of k variables appears in the RSE, all products of k variables appear.As our inductive hypothesis we assume that all products of 3k variables are absent inthe RSE but all products of 3k � 1 and 3k � 2 variables are present. This holds fork = 1. We assume it holds for k � l and show it holds for k = l + 1. There are Cn

j


products of j variables. Let P (j) be 1 if products of j variables are in the RSE and letit be 0 otherwise. It follows that0@j=mX

j=0

P (j)Cmj

1A mod 2 =

(1 f

(n)mod3 = 1

0 f(n)mod3 = 0

when m is the number of 1’s among the n variables. We use this identity to show byinduction that P (j) is 0 when j = 3k and 1 otherwise. Note that if P (j) = 1 for allvalues of j, the sum on the left-hand side is 2m. Given our inductive hypothesis, wehave the following three identities:

j=mXj=0

P (j)Cmj = 2m + 1�

kXj=0

C3k3j � P (m)

for m = 3k+1, 3k+2, 3k+3. Since the sum of the binomial terms is even, it followsthat 0@j=mX

j=0

P (j)Cmj

1A mod 2 = (1 � P (m)) mod 2

Since f (n)mod3 is 0 when m = 3k + 1, 3k + 2 but 1 when m = 3k + 3, we have thatP (3k + 1) = 1, P (3k+ 2) = 1 but P (3k + 3) = 0.

2.11 Problem: Show that every Boolean function f (n) : Bn 7! B can be expanded asfollows:

f(x1,x2, : : : ,xn) = x1f(1,x2, : : : ,xn) _ x1f(0,x2, : : : ,xn)

Apply this expansion to each variable of f(x1,x2,x3) = x1x2 _ x2x3 to obtain itsDNF.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: To establish that every Boolean function f (n) : Bn 7! B has this expansionwe show that the left and right-hand sides have the same value when x1 = 1 andx1 = 0, which is clearly the case.Applying this expansion to f(x1,x2,x3) = x1�x2 _ x2x3 we generate the followingsequence of expansions, the last of which is its DNF.

f(x1,x2,x3) = x1�x2 _ x2x3

= x1(�x2 _ x2x3) _ x1(x2x3)

= x1(x2(x3) _ x2(1)) _ x1x2x3

= x1x2x3 _ x1x2 _ x1x2x3

= x1x2x3 _ x1x2(x3 _ x3) _ x1x2x3

= x1x2x3 _ x1x2x3 _ x1x2x3 _ x1x2x3

2.12 Problem: In a dual-rail logic circuit 0 and 1 are represented by the pairs (0, 1) and(1, 0), respectively. A variable x is represented by the pair (x,x). A NOT in thisrepresentation (called a DRL-NOT) is a pair of twisted wires.

2.1 Solutions 23

a) How are AND (DRL-AND) and OR (DRL-OR) realized in this representation? Usestandard AND and OR gates to construct circuits for gates in the new representa-tion. Show that every function f : Bn 7! Bm can be realized by a dual-rail logiccircuit in which the standard NOT gates are used only on input variables (to obtainthe pair (x, �x)).

b) Show that the size and depth of a dual-rail logic circuit for a function f : Bn 7! Bare at most twice the circuit size and at most one more than the circuit depth of fover the basis fAND, OR, NOTg when the n NOTs on inputs are not counted.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: Let x and y denote either input variables or the values of two gates in acircuit for f . If x and y are represented by (x, �x) and (y, �y), respectively, their AND isrepresented by (x^y,x ^ y), that is, by (x^y, �x_�y). Thus, the DRL-AND is realizedfrom (x, �x) and (y, �y) by a circuit containing one AND and one OR. Similarly, the OR

of the two variables is represented by (x_ y,x _ y) or, equivalently, by (x_ y, �x ^ �y).Thus, the DRL-OR is realized by a circuit with one AND and one OR. Since thisconstruction only adds depth at the input (to compute the negation of inputs), andat most doubles the number of ANDs and ORs (negations do not require gates), thefollowing bounds apply:

CDRL(f) � 2C(f)

DDRL(f) � D(f) + 1

2 Problem: A function f : Bn 7! B is monotone if for all 1 � j � n, f(x1, : : : ,xj�1,0,xj+1, : : : ,xn) � f(x1, : : : ,xj�1, 1,xj+1, : : : ,xn) for all values of the remainingvariables; that is, increasing any variable from 0 to 1 does not cause the function todecrease its value from 1 to 0.

a) Show that every circuit over the basis mon = fAND, ORg computes monotonefunctions at every gate.

b) Show that every monotone function f (n) : Bn 7! B can be expanded as follows:

f(x1,x2, : : : ,xn) = x1f(1,x2, : : : ,xn) _ f(0,x2, : : : ,xn)

Show that this implies that every monotone function can be realized by a logic circuitover the monotone basis mon = fAND, ORg.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) We give a proof by induction. Clearly, AND and OR gates compute monotonefunctions. Assume that every circuit containing g � 1 or fewer gates over the basismon computes monotone functions at every gate. Consider the output gate ofa circuit over this basis that has g gates. Since both of its inputs are the values ofmonotone functions and the gate itself is monotone, the function computed by thecomplete circuit is monotone.

b) Because f (n) : Bn 7! B is monotone, if f(0,x2, : : : ,xn) = 1, then f(1,x2, : : : ,xn)is also 1. Thus, the left- and right-hand sides of the following equation both have


the same value when x1 = 0 and when x1 = 1.

f(x1,x2, : : : ,xn) = x1f(1,x2, : : : ,xn) _ f(0,x2, : : : ,xn)

SPECIALIZED FUNCTIONS

2.14 Problem: Complete the proof of Lemma 2.5.3 by solving the recurrences stated inEquation (2.4).. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Let �(n) = C0

�f(n)encode

�and �(n) = D0

�f(n)encode

�. Then, �(1) =

�(1) = 0 because f (n)encode(x1) = (x1,x1) and we do not count negations. The recur-rences in question are the following:

�(n) � 2n � 1 + �(n � 1) (2.1)

�(n) � max(n� 1, �(n� 1) + 1) (2.2)

We assume as inductive hypotheses the following for m � n� 1:

�(m) � 2m+1 � (m+ 3)

�(m) � m� 1

These inequalities are satisfied for the base case. It is straightforward to show that theinductive hypothesis holds for m = n.

2.15 Problem: Design a multiplexer circuit of circuit size 2n+1 plus lower-order terms whenn is even.Hint: Construct a smaller circuit by applying the decomposition given in Section 2.5.4of the minterms of n variables into minterms on the two halves of the n variables.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Output variable yj of the decoder is associated with a minterm x(c) wherej = jcj. In addition, x(c) can be written as the product of two minterms u(a) andv(b) where the n-tuple x is decomposed as an n=2-tuple u and an n=2-tuple v. Theconstant n-tuple c is decomposed into corresponding n=2-tuples a and b. Thus, jcj =2n=2jaj+ jbj. This leads to the following representation of f (n)mux:

f (n)mux(x,y) =_a,b

u(a) ^ v(b) ^ yjcj

=_a

u(a ) ^

_b

v(b) ^ y2n=2jaj+jbj

!!

Let the minterms fv(a)g and fv(b)g be realized by two decoder circuits computing

f(n=2)decode. Each of the 2n=2 inner sums (indexed by a) can be formed with 2n=2 ^’s and

2n=2�1 _’s for a total of 2n+1�2n=2 additional two-inputs gates and depth (n=2)+1.The outer sum can be formed with 2n=2 additional^’s and 2n=2�1 additional_’s and

2.1 Solutions 25

additional depth (n=2) + 1. The depth of the resulting circuit is n+ 2 plus the depthof a circuit for the minterms fv(b)g. Using the bounds given in Lemma 2.5.4 we

have that for n divisible by four the multiplexer function f(n)mux : B2n+n 7! B can be

realized with the following circuit size and depth over the basis 0 = f^,_,:g :

C0(f(n)mux) � 2n+1 + 3 � 2(n=2) + (2n� 4)2(n=4) � 1

D0(f(n)mux) � n+ dlog2 ne+ 3

2.16 Problem: Complete the proof of Lemma 2.11.1 by establishing the correctness of theinductive hypothesis stated in its proof.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The recurrences of the proof are shown below.

D(c(l+1)j+1 ) = max(D(c

(l+1)j ) + 2,D(s

(l)j ) + 3) (2.3)

D(s(l+1)j ) = max(D(c

(l+1)j ) + 1,D(s

(l)j ) + 2) (2.4)

for 2 � l and 0 � j � l � 1 where s(l)l�1 = c(l)l�1. The solutions to these recurrences

are the following which we establish by induction on k and j for 2 � k:

D(c(k)r ) = 2(k + r) � 3 1 � r � k � 1 (2.5)

D(s(k)t ) = 2(k + t) � 2 0 � t � k � 2 (2.6)

The base case is k = 2 for which we have D(s(2)0 ) = 2 and D(c

(k)1 ) = 3 by

examination of Fig. 2.21(a). It is straightforward to show that D(s(k)0 ) = 2k � 2 for

all k by noting that D(c(k)0 ) = 0 for all k � 2 and applying (2.4). It follows from

(2.3) by induction that D(c(k)1 ) = 2k � 1.

We show that (2.5) and (2.6) hold by induction on k as follows. The base case is k = 2.Assume that it holds for all values of r and t in the indicated ranges for k = m�1. Weshow it holds for k = m by inducting on l for r = l + 1 and t = l in (2.5) and (2.6).The base case is l = 0 which is established in the previous paragraph. Assume (2.5)and (2.6) hold for l = p � 1, 1 � p � k � 1 and show that it holds for l = p. Butthis follows directly from substitution in (2.3) and (2.4). This completes the doubleinduction.

2.17 Problem: The binary sorting function is defined in Section 2.11. Show that it can berealized with a circuit whose size is O(n) and depth is O(logn).

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The binary sorting f(n)sort : Bn 7! Bn has the following size and depth

bounds where k = dlog(n+ 1)e

C(f(n)sort) � 9 2k � 5k � 11 + k

p2k+1 � 2

D(f(n)sort) � 6k � 4 + dlog2 ke


To show this, choose 2k�1�1 < n � 2k�1 and construct a circuit for these functionson 2k � 1 inputs by filling in the remaining variables with 0. These are stripped offlater.

The proof of Theorem 2.11.1 gives a construction for the elementary symmetric func-tions for m = 2k � 1 which can be translated into a circuit for the threshold functionson the same number of inputs (the binary sorting function) by performing a prefixcomputation on the elementary symmetric functions with the OR function. That is,�(m)m = e

(m)m , � (m)

m�1 = e(m)m�1 _ e

(m)m , : : : , � (m)

1 = e(m)1 _ � � � _ e

(m)m�1 _ e

(m)m .

From Theorem 2.6.1 it follows that with 2m� 2 additional ORs and additional depth2dlog2 me + 1 the sorting function on m, inputs can be realized.

LOGICAL FUNCTIONS

2.18 Problem: Let f (n)member : B(n+1)b 7! B be defined below.

f(n)member(x1,x2, : : : ,xn,y) =

(1 xi = y for some 1 � i � n

0 otherwise

where xi,y 2 Bb and xi = y if and only if they agree in each position.

Obtain good upper bounds to C

�f(n)member

�and D

�f(n)member

�by constructing a

circuit over the basis = f^,_,:,�g.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Consider the function f(n)match : B2n 7! B where f (b)match(x,y) has value 1

only when x = y. The following is a formula for f (n)match because (xi � yi) has value 1only when xi and yi have the same value.

f(b)match(x,y) = (x1 � y1) ^ (x2 � y2) ^ � � � ^ (xb � yb)

Applying DeMorgan’s rule this can be written as the NOT of the OR of the terms

(xi � yi) for 1 � i � n. The function f(b)match can be realized by a circuit with b

EXCLUSIVE OR’s, one NOT, b� 1 ORs or a total of 2b gates and depth dlog2 be + 2.

Also, we can write f (n)member as follows:

f(n)member(x1,x2, : : : ,xn,y) = f

(b)match(x1,y) _ f (b)match(x2,y) _ � � � _ f (b)match(xn,y)

Thus, the function f (n)member can be realized with n instances of f (b)match and an OR-treeon n inputs having n � 1 two-input OR’s and depth dlog2 ne. Thus, we have thefollowing size and depth bounds on f (n)member over the basis = f^,_,:,�g:

C(f(n)member) � 2nb+ n� 1

D(f(n)member) � dlog2 be+ 2 + dlog2 ne

2.19 Problem: Design a circuit to compare two n-bit binary numbers and return the value1 if the first is larger than or equal to the second and 0 otherwise.

2.1 Solutions 27

Hint: Compare each pair of digits of the same significance and generate three out-comes, yes, maybe, and no, corresponding to whether the first digit is greater than,equal to or less than the second. How can you combine the outputs of such a compar-ison circuit to design a circuit for the problem? Does a prefix computation appear inyour circuit?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Let the two binary numbers be x = (xn�1,xn�2, : : : ,x0) and y = (yn�1,yn�2, : : : , y0). Let g(x, y) = (zy , zm) where zy = x�y and zm = xy _ �x �y. Then,zy = 1 when x > y and zm = 1 when x = y; that is, (zy , zm) characterize two ofthe three possible outcomes. The third result can be calculated from these two sincezn = zy _ zm.

Let (z(j)y , z(j)m ) = g(xj , yj). Then z(j)y = 1 if the number corresponding to the j leastsignificant bit of x is larger than that corresponding to the jth least significant bit of y.Also, z(j)m = 1 if jth most significant bits of the two numbers are equal.Let (k)(x,y) have value 1 if the integer denoted by bits 0 through k of x is largerthan that of y and 0 otherwise. We can write the following recurrence for (n):

(n�1) = z(n�1)y _ z(n�1)

m (n�2)

Also, (0) = z(0)y . Thus, we can write the following expression for (n).

(n�1) = z(n�1)y _ z(n�1)

m z(n�2)y _ z(n�1)

m z(n�2)m z(n�3)

y _ z(n�1)m z(n�2)

m z(n�3)m z(n�4)

y

_ z(n�1)m z(n�2)

m � � �z(1)m z(0)y

It follows that (n�1) can be computed by first performing the prefix computation

(1, z(n�1)m , z(n�1)

m z(n�2)m , : : : , z(n�1)

m z(n�2)m � � � z(1)m ), ANDing the results with the “yes”

values z(n�1)y , z(n�2)

y , : : : , z(0)y , and ORing the results.

PARALLEL PREFIX

2.20 Problem:

a) Let �copy : S2 7! S be the operation

a�copy b = a

Show that (S,�copy) is a semigroup for S an arbitrary non-empty set.b) Let � denote string concatenation over the set f0, 1g� of binary strings. Show that

it is associative.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution:

a) We must show that �copy is associative. This holds if for three values a, b and c inS we have:

(a�copy b)�copy c = a�copy (b�copy c)Since a�copy b = a and b�copy c = b, the identity follows.


b) This result follows because (a � b) � c = a � (b � c).2.21 Problem: The segmented prefix computation with the associative operation � on a

“value” n-vector x over a set S, given a “flag vector” � over B, is defined as follows: thevalue of the ith entry yi of the “result vector” y is xi if its flag is 1 and otherwise is theassociative combination with � of xi and the entries to its left up to and including thefirst occurrence of a 1 in the flag array. The leftmost bit in every flag vector is 1. Anexample of a segmented prefix computation is given in Section 2.6.Assuming that (S,�) is a semigroup, a segmented prefix computation over the setS � B of pairs is a special case of general prefix computation. Consider the operator on pairs (xi,�i) of values and flags defined below:

((x1,�1) (x2,�2)) =

((x2, 1) �2 = 1

(x1 � x2,�1) �2 = 0

Show that ((S,B),) is a semigroup by proving that (S,B) is closed under the oper-ator and that the operator is associative.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: The operation is closed because (S,�) is a semigrup. So we need only showthat is associative over S � B. Consider three pairs (x1,�1), (x2,�2) and (x3,�3)in S � B. We have that

((x1,�1) (x2,�2)) (x3,�3) =

((x2, 1) (x3,�3) �2 = 1

(x1 � x2,�1) (x3,�3) �2 = 0

=

8><>:(x3, 1) �3 = 1

(x2 � x3, 1) �2 = 1 and �3 = 0

(x1 � x2 � x3,�1) �2 = 0 and �3 = 0

On the other hand the following holds and the operation is associative.

(x1,�1 ((x2,�2) (x3,�3)) =

((x1,�1) (x3,�3) �3 = 1

(x1,�1) (x2 � x3,�2) �3 = 0

=

8><>:(x3, 1) �3 = 1

(x2 � x3, 1) �2 = 1 and �3 = 0

(x1 � x2 � x3,�1) �2 = 0 and �3 = 0

2.22 Problem: Construct a logic circuit of size O(n logn) and depth O(log2 n) that, givena binary n-tuple x, computes the n-tuple y containing the running sum of the numberof 1’s in x.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Since integer addition is associative, the running sum of the 1’s in x can becomputed by a prefix circuit in which the operator is an adder. Since the maximumnumber of 1’s is n, the adder need only be designd for (log2 n)-bit numbers. A carry

2.1 Solutions 29

lookahead adder for numbers of such length has sizeO(logn) and depth O(log logn).A prefix circuit to perform the addition operation has O(n) adders in a circuit in whichthe longest path contains O(logn) adders. Thus, this problem can be solved by acircuit of size O(n logn) and depth O(logn log logn).

2.23 Problem: Given 2n Boolean variables organized as pairs 0a or 1a, design a circuit thatmoves pairs of the form 1a to the left and the others to the right without changing theirrelative order. Show that the circuit has size O(n log2 n).

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: We sketch a circuit to move the pairs 0a to the right. A parallel circuit canbe constructed to move pairs 1a to the left.

Let there be n positions holding pairs xa. Number the positions 1 to n from leftto right. Consider a pair in position k. A prefix computation is done from right toleft that assigns to position k an integer which is the number of pairs with 0 as thefirst component to its right including itself. In Problem 2.22 we describe a circuit tocompute a running sum of 1’s which can be used for this purpose. If position k has asum rk and holds the rkth 0-pair from the right, the quantity sk = n � k + 1 � rkspecifices the number of positions that this pairs needs to shift right in order to arrivein its final position.

Represent sk as a binary number. We now design shifting circuits to move each pairs(and a corresponding shift value) to the right. The first shifting circuit shifts the kthpair 0 places if the least significant bit of sk is 0 and 1 place if it is 1. We shift sk fromwhich the least significant bit is deleted along with the kth pair. We repeat this processwith a circuit that shifts the kth pair by 0 or 2 places depending on whether the secondleast significant bit of sk is 0 or 1. Along with the kth pair, we carry sk from which thetwo least significant bits are dropped. This process is repeated with shifters that shift byincreasing powers of two. It is not hard to show that this process does not result in twoor more pairs arriving at the same location at the same time.

A circuit constructed in this fashion has size O(n log2 n) because each shifting stagecan be realized by a circuit of size O(n logn) and there are logn stages and becausethe computation of the quantities sk for 1 � k � n can be done with a circuit of sizeO(n logn).

2.24 Problem: Linear recurrences play an important role in many problems including thesolution of a tridiagonal linear system of equations. They are defined over “near-rings,”which are slightly weaker than rings in not requiring inverses under the addition oper-ation. (Rings are defined in Section 6.2.1.)

A near-ring (R, �,+) is a set R together with an associative multiplication operator �and an associative and commutative addition operator +. (If + is commutative, thenfor all a, b 2 R, a + b = b + a.) In addition, � distributes over +; that is, for alla, b, c 2 R, a � (b+ c) = a � b+ a � c.A first-order linear recurrence of length n is an n-tuple x = (x1,x2, : : : ,xn) ofvariables over a near-ring (R, �,+) that satisfies x1 = b1 and the following set ofidentities for 2 � j � n defined in terms of elements faj, bj 2 R j 2 � j � ng:

xj = aj � xj�1 + bj


Use the ideas of Section 2.7 on carry-lookahead addition to show that xj can be written

xj = cj � x1 + dj

where the pairs (cj, dj) are the result of a prefix computation.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: We can solve for x3 in terms of x1 by substituting the value for x2 into itsrecurrence. Similarly, for j � 4 we can substitute the value of xj�1 into the recurrencefor xj. Assume we can write xj for 2 � j � r � 1 in terms of x1 and coefficients(cj , dj), as shown below.

xj = cjx1 + dj

Clearly (c2, d2) = (a2, b2). Substituting xr�1 into the recurrence for xr we have

xr = cr � x1 + dr (2.7)

where

cr = ar � cr�1

dr = br + ar � dr�1

We can treat (cr, dr) as the result of a new product operation � that combines the pairs(ar , br) and (cr�1, dr�1), as indicated below.

(cr , dr) = (ar, br) � (cr�1, dr�1)

Restated in generic terms this equation becomes

(x1, y1) � (x2, y2) = (x1 � x2, y1 + x1 � y2)

To show that this operation � is associative we consider three pairs (x1, y1), (x2, y2),and (x3, y3) and their combinations shown below.

((x1, y1) � (x2, y2)) � (x3, y3) = (x1 � x2, y1 + x1 � y2) � (x3, y3)

= ((x1 � x2) � x3, y1 + x1 � y2 + (x1 � x2) � y3)

= (x1 � (x2 � x3), y1 + x1 � (y2 + (x2 � y3))

= (x1, y1) � ((x2, y2) � (x3, y3))

Here we have used the associativity of � and the associtivity of + and the fact that it dis-tributes over �. Note that the proof of associativity of � for first-order linear recurrencesdoes not require commutativity of the addition operator +.

Since the set S of pairs f(a, b)g for a, b 2 R under the operation � is closed, (S, �)forms a semigroup. Thus, the prefix computation on the pairs ((a2, b2) , (a3, b3) , : : : ,(an, bn)) produces the pairs ((c2, d2) , (c3, d3) , : : : , (cn, dn)). From these pairs andx1 the values (x2,x3, : : : ,xn) can be computed from (2.7).

2.1 Solutions 31

ARITHMETIC OPERATIONS

2.25 Problem: Design a circuit that finds the most significant non-zero position in an n-bitbinary number and logically shifts the binary number left so that the non-zero bit is inthe most significant position. The circuit should produce not only the shifted binarynumber but also a binary representation of the amount of the shift.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Let the n-bit binary number be x = (xn�1, xn�2, : : : , x0). From thisnumber we generate a vector y such that yj = 0 for j 6= k and yk = 1 if xk is themost significant non-zero digit of x. Consider tk = xn�1 _ � � � _ xk+1. It is 0 if nohigher order position than the kth contains a 1 and 1 otherwise. Thus, if tk = 0 andxk = 1 then yk = 1. Otherwise, yk = 0. Thus, yk = tk ^ xk. Thus, to compute ykfor 0 � k � n�1 we can perform the prefix computation on the vector (xn�1, xn�2,: : : , x0) with the OR operation to produce (tn�1, tn�2, : : : , t0) (note that tn�1 = 0)after which we combine tk and xk as indicated to produce y.

Once y has been produced we convert from this positional notation to binary notationusing the encoder circuit of Section 2.11.1 and then use this result as the value of theamount of shift in a logical shifting circuit.

The size and depth of the circuit to compute y are O(n) and O(logn). The samebounds apply to the encoder circuit. The shifting circuit has sizeO(n logn) and depthO(logn). Thus, the combined circuit has size O(n logn) and depth O(logn).

2.26 Problem: Consider the function �[j, k] = �[j, k�1]��[k, k] for 1 � j < k � n�1,where � is defined in Section 2.7.1. Show by induction that the first component of�[j, k] is 1 if and only if a carry propagates through the full adder stages numberedj, j + 1, : : : , k and its second component is 1 if and only if a carry is generated at oneof these stages, propagates through subsequent stages, and appears as a carry out of thekth stage.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The carry out of the jth stage, cj+1, is defined in terms of the carry in andthe propagate and generate bits as follows:

cj+1 = pjcj _ gj

Substituting into this expression that for cj+2 we have

cj+2 = pj+1pjcj _ pj+1gj _ gj+1

Clearly, a carry is propagated through stage j and j + 1 if pj+1pj = 1 and generatedin these stages if pj+1gj _ gj+1 = 1. This is the base case for induction. Given that �is associative, as stated in Section 2.7.1, we need only show to show that if �[j, k � 1]satisfies the inductive hypothesis, then so does �[j, k] = �[j, k � 1] � �[k, k]. But thisfollows from the above analysis.

2.27 Problem: Give a construction of a circuit for subtracting one n-bit positive binaryinteger from another using the two’s-complement operation. Show that the circuit hassize O(n) and depth O(logn).


. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The 2’s complement of an n-bit binary number is obtained by complement-ing each bit and adding 1 to the result. To add a negative number x to a positivenumber y, complement each bit of x with NOT gates, and add the complement of x,y, and 1 using a carry lookahead adder to which 1 is supplied as the carry in to the firststage. The combined circuit has size O(n) and depth O(logn).

2.28 Problem: Complete the proof of Theorem 2.9.3 outlined in the text. In particular,solve the recurrence given in equation (2.10).

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: We have the following two recurrences in which C(1) = D(1) = 1 andc, d � 1.

C(n) � 3C(n=2) + cn

D(n) � D(n=2) + d log2 n

When n = 2k, let �(k) = C(n) and Æ(k) = D(n). We claim that �(k) � c(3k+1 �2k+1). The first claim holds when k = 0 since c � 1 and for larger k as can be seenby straightforward substitution. We claim that Æ(k) � d(k(k+ 1)=2). The first claimholds when k = 0 since d � 1 and for larger k as can be seen by straightforwardsubstitution.

2.29 Problem: Show that the depth bound stated in Theorem 2.9.3 can be improved fromO(log2 n) to O(logn) without affecting the size bound by using carry-save additionto form the six additions (or subtractions) that are involved at each stage.

Hint: Observe that each multiplication of (n=2)-bit numbers at the top level is ex-panded at the next level as sums of the product of (n=4)-bit numbers and that this typeof replacement continues until the product is formed of 1-bit numbers. Observe alsothat 2n-bit carry-save adders can be used at the top level but that the smaller carry-saveadders can be used at successively lower levels.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: This algorithm has O(logn) stages of addition each of which is a carry-saveaddition. The output of the last stage is a pair of binary numbers whose sum is thedesired result. Put these last two binary numbers into a carry lookahead adder. Sinceeach of the O(logn) stages except that last has a fixed depth and the last has depthO(logn), the depth of this circuit is O(logn).

2.30 Problem: Residue arithmetic can be used to add and subtract integers. Given positiverelatively prime integers p1, p2, : : : , pk (no common factors), an integer n in the setf0, 1, 2, : : : ,N � 1g, N = p1p2 � � �pk, can be represented by the k -tuple n =(n1,n2, : : : ,nk), where nj = n mod pj . Let n and m be in this set.

a) Show that if n 6= m, n 6=m.

b) Form n +m by adding corresponding jth components modulo pj. Show thatn+m uniquely represents (n +m) mod N .

c) Form n �m by multiplying corresponding jth components of n and m modulopj. Show that n �m is the unique representation for (nm) mod N .

2.1 Solutions 33

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) We show that if n 6= m, n 6=m by contradiction. If n =m, then (ni �mi) =(n�m) mod pi = 0 for 1 � i � k. Since (n�m) is divisible by p1, p2, : : : , pk,it is divisible by their product N . Since n � m lies between -(N-1) and N-1,the only integer in this range divisible by N is 0 and n = m, contradicting theassumption that n 6= m.

b) Write n+m = qN +(n+m) mod N where q is the quotient on dividingn+m

by N . The remainder (n +m) mod N has value between 0 and N � 1. The ithcomponent of n+m is

(n mod pi +m mod pi) mod pi = (n+m) mod pi

= (qN + (n+m) mod N ) mod pi

= (n+m) mod N mod pi

The latter follows because N is divisible by pi. By a) this representation is unique.c) Let n�m denote the tuple obtained by multiplying the corresponding jth com-

ponents of n and m modulo pj . The proof of this result is analogous to that forb), namely, we observe that we can write n = qnpi + (n mod pi) (and a similarexpression for m). Thus,

nm = p2iqnqm + pi(qn(m mod pi) + qm(n mod pi)) + (m mod pi)(n mod pi)

It follows that nm mod pi is the product (m mod pi)(n mod pi) mod pi. Be-cause we can write nm = qN + (nm mod N ) and N is divisible by pi, n�mis the unique representation for (nm) mod N .

2.31 Problem: Use the circuit designed in Problem 2.19 to build a circuit that adds twon-bit binary numbers modulo an arbitrary third n-bit binary number. You may useknown circuits.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Let the three numbers be x, y, and z. The sum of the first two n-bitnumbers cannot exceed 2n+1 � 1. Let z be fixed. To compute the remainder of thissum modulo z, subtract z at most k = d(2n+1 � 1)=ze times from the sum. Thus,a circuit for the problem in question can be constructed from an adder the output towhich is passed through k circuits each of which compares the result of the previousstage with z using the circuit Problem 2.19 and subtracts z if it is larger. Subtractionis done by adding the two’s complement of z.What kind of circuit should be constructed when z is not known in advance? Howabout a division circuit?

2.32 Problem: In prime factorization an integer n is represented as the product of primes.Let p(N ) be the largest prime less than N . Then, n 2 f2, : : : ,N � 1g is representedby the exponents (e2, e3, : : : , ep(N)), where n = 2e23e3 : : : p(N )ep(N) . The represen-tation for the product of two integers in this system is the sum of the exponents of theirrespective prime factors. Show that this leads to a multiplication circuit whose depthis proportional to log log logN . Determine the size of the circuit using the fact thatthere are O(N= logN ) primes in the set f2, : : : ,N � 1g.


. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The largest exponent in the prime factorization of any integer in the set is atmost log2 N . When represented in binary such numbers require O(log logN ) bits. Itfollows that exponents can be added with carry lookahead adders with O(log logN )gates per exponent for a total of O(N log logN= logN ) gates. The depth of each ofthese adders is O(log log logN ).

2.33 Problem: Construct a circuit for the division of two n-bit binary numbers from cir-cuits for the reciprocal function f

(n)recip and the integer multiplication function f

(n)mult.

Determine the size and depth of this circuit and the accuracy of the result.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: To divide x by y, form the reciprocal of y and multiply by x. The sizeand depth of the resulting circuit is the sum of the sizes and depths of circuits for thereciprocal and multiplication functions.The reciprocal r of the integer u (we assume that u � 2n�1) satisfies ru = 22n � s

where 0 � s � u. Thus, the product v with r satisfies vr = (v=u)22n � vs. Thus,the error in the result is at most vs. When normalized by dividing 22n, it is at mostuv=22n.

2.34 Problem: Let f : Bn 7! Bkn be an integer power of x; that is, f(x) = dxke for

some integer k. Show that such functions contain the shifting function f(m)shift as a

subfunction for some integer m. Determine m dependent on n and k.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Let n be divisible by 4. Consider first the case k = 2. Since x is a binary n-tuple, decompose it into two (n=2)-tuples, a high-order (n=2)-tuple b and a low-order(n=2)-tuple a. We can then write the corresponding integer jxj as follows:

jxj = 2n=2jbj+ jajThe function f(x) can be written as follows:

f(x) = 2njbj2 + 2n=2+1jbjjaj+ jaj2

Now let jbj = 2t for n=4 � t � n=2� 1 and let jaj � 2n=4 � 1. It follows that themiddle term in the expansion of f(x) contains n=4 bits that are free to vary and theydo not overlap the bits in the high order or low order terms. The “middle” n=4 bits areshifted by 0 through n=4� 1 positions. Thus, m = n=4.To generalize this result for k which is a power of two, expand jxj as follows:

jxj = 2ljbj+ jajfor jaj � 2l�1 and jbj � 2n�l�1. Use the binomial theorem to expand (jbj2l+jaj)kand choose jaj and jbj such that the term kjbjk�1jaj does not overlap with any otherterm in the expansion. When jbj is a power of 2, this term represents a shift of jaj byk � 1 places. To meet these conditions we require that kjbjk�1jaj be less than jajkand more than

�k2

�jbjk�2jaj2. Both conditions are satisfied if kjaj < jbj. If we choosejaj � 2l=2 � 1 and jbj � 2(n�l)=2, the inequality holds for l � (n=2) � dlog2 ke.Since the multiplier of jaj in kjbjk�1jaj is kjbjk�1, when jbj is a power of two, jaj isshifted in steps of size k � 1. Thus, m = b(n � (n � l))=(k � 1)c = bl=(k � 1)c.

2.1 Solutions 35

2.35 Problem: Let f : Bn 7! Bn be a fractional power of x of the form f(x) = dxq=2ke,0 < q < 2k < log2 n. Show that this function contains the shifting function f (m)

shift asa subfunction. Find the largest value of m for which this holds.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Analogous to the previous problem, the approach is to compute a Taylorseries expansion of the function.

C H A P T E R

Machines with Memory


3.1 Problem: Establish the following identity:

S(k) =kXj=0

j2j = 2�(k � 1)2k + 1

�. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The proof is by induction. The result holds for k = 0. Assume it holds fork � m� 1 and show that it holds for k = m. The following relationship follows fromthe definition.

S(m) = S(m � 1) +m2m = 2�(m � 2)2m�1 + 1

�+m2m

The conclusion follows by simple algebraic manipulation.

3.2 Problem: Let p :N 7!N and q :N 7!N be polynomial functions on the setN ofnon-negative integers. Show that p(q(n)) is also a polynomial in n.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Since p and q are polynomial functions, there exist integers r and s suchthat p(n) � a1n

r + b1 and q(n) � a2ns + b2 for positive integers a1, a2, b1, b2.

Then, p(q(n)) � a1(a2ns + b2)

r + b1. Since (anr + b)s � (a+ b)snrs, the desiredconclusion follows.

37

38 Chapter 3 Machines with Memory

FINITE-STATE MACHINES

3.3 Problem: Describe an FSM that compares two binary numbers supplied as concurrentstreams of bits in descending order of importance and enters a rejecting state if the firststring is smaller than the second and an accepting state otherwise.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The FSM has accept and reject states, qa and qr. Once it enters one of thesestates, it returns to it on all inputs. It also has a “maybe” state, qm, which is also theinitial state. If the FSM is in qm and the first of the two input bits is 1 and the other0, it enters qa. It enters qr if the first bit is 0 and the second 1. In the remaining cases(both inputs are the same), it returns to qm.

3.4 Problem: Describe an FSM that computes the threshold-two function on n Booleaninputs that are supplied sequentially to the machine.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: This FSM has three states, q0, the initial state and that state the FSM is in ifit all prior input bits are 0, q1, the state in which it resides if it has seen a single 1 amongits input bits, and q2, the (final) state it enters if it has seen two or more 1’s among itsinput bits. If in q0 the FSM receives an input of 1, it enters q1. Otherwise it returns toq0. If in q1 the FSM receives an input of 1, it enters q2. Otherwise it returns to q1. Inq2 the FSM returns to q2 on all inputs.

3.5 Problem: Consider the full-adder function fFA(xi, yi, ci) = (ci+1, si) defined belowwhere + denotes integer addition:

2ci+1 + si = xi + yi + ci

Show that the subfunction of fFA obtained by fixing ci = 0 and deleting ci+1 is theEXCLUSIVE OR of the variables xi and yi.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The value of the output s1 is 1 if the number of 1’s among the three inputxi, yi, and zi is odd. This is exactly the parity function.

3.6 Problem: It is straightforward to show that every Moore FSM is a Mealy FSM. Givena Mealy FSM, show how to construct a Moore FSM whose outputs for every inputsequence are identical to those of the Mealy FSM.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Given a Mealy FSM M = (�,,Q, Æ,�, s,F ), we construct a Moore FSMM 0 = (�,,Q � , Æ0,�0, s0,F 0). In M 0, each state is a pair [q=b], q a state of Mand b an output symbol that can occur on a transition into state q. [qi=bj] 2 F 0, thefinal states of M 0, if qi 2 F . The initial state of M is s0 = [s=bk] for bk an arbitraryoutput letter of M .

For every c 2 let Æ0([q=c], a) = [q0=b0] where q0 = Æ(q, a) and b0 = �(q, a).Also, let �0([q0=b0]) = b0. Then, if output b0 is produced on a transition by the Mealymachine M to state q0, the Moore machine M 0 enters state [q0=b0] and produces thesame output. The state [q0=b0] captures the fact that M has entered state q0.Because the successor to M 0’s state [q=c] is the same for every output symbol c, eachtransition from [q=c] by M 0 corresponds to a transition from state q by M .

3.1 Solutions 39

qa qb qc qd qe qfStart

0 0 1 0 1

1 0

1

1 1

00

Figure S.3.1 The figure associated with Problem 3.7.

We show by induction that M 0 produces the same output sequence for a given inputsequence as does M . The statement is true for input sequences of length one whenthe two machines are in their initial states. We assume that it is true for states thatare reachable from initial states by input sequences of length k or less and show that itholds for input sequences of length k+1. Since for every output letter c, the transitionfrom state [q=c] of M 0 on input a generates the same output as does M in state q withinput a, the conclusion follows.

3.7 Problem: Find a deterministic FSM that recognizes the same language as that recog-nized by the nondeterministic FSM of Fig. 3.8.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The method of Section 4.2 is used to construct the FSM shown in Fig-ure S.3.1. Here qa = fq0g, qb = fq0, q1g, qc = fq0, q1, q2g, qd = fq0, q3g,qe = fq0, q1, q4g, and qf = fq0, q5g.

3.8 Problem: Write a program in a language of your choice that writes the straight-lineprogram described in Fig. 3.3 for the FSM of Fig. 3.2 realizing the EXCLUSIVE OR

function.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The next-state/output function for the FSM of Fig. 3.2 is given in Equa-tion 2.2. We need only write a program that writes this equation T times, T an inputparameter, that changes the the numbers of the statements in this equation so that everystatement is given a unique number.

SHALLOW FSM CIRCUITS

3.9 Problem: Develop a representation for states in the m-word, b-bit random-accessmemory so that its next-state mappings form a semigroup.Hint: Show that the information necessary to update the current state can be succinctlydescribed.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Consider the random-access memory unit withM b-bit words and one b-bitoutput word. It has 2(M+1)b states. There is one next-state mapping for the no-opcommand; it maps the current state to itself. There are M2b next-state mappingscorresponding to writing a word in memory. For each such mapping the contents ofone word is replaced by a new word. Thus, all states differing only in the word that


is replaced have the same successor state. Finally, the read command maps the currentstate to the state that differs only in the value of the output word.

3.10 Problem: Show that matrix multiplication is associative.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The goal is to show that A(BC) = (AB)C where A, B, and C are n� n

matrices. Applying the definition of matrix multiplication twice and using the associa-tivity of addition and multiplication over a ring, we have the following derivation:

(A(BC))r,s =nXi=1

ar,i(BC)i,s

=nXi=1

ar,i(nXj=1

bi,jcj,s)

=nXj=1

(nXi=1

ar,ibi,j)cj,s

=nXj=1

(AB)r,jcj,s

= ((AB)C)r,s

SEQUENTIAL CIRCUITS

3.11 Problem: Show that the circuit of Fig. 3.15 computes the functions defined in thetables of Fig. 3.14.Hint: Section 2.2 provides a method to produce a circuit from a tabular description ofa binary function.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: We can use the hint and design a circuit from the tables for Æ and �. Thefunction � is trivial because it produces as output the value of the second componentin the representation (c, s) for the state of the FSM. By inspection the value of thefunction Æ on the state is the same for both values of the input s. Thus, it dependson the values of c, u, and v. Again by inspection, the value of (c�, s�) is the binaryrepresentation of the number of 1’s among the three inputs c, u, and v. Since, as shownin Section 2.7, this function is computed by the Full Adder circuit, the sequential circuitof Fig. 3.15 does realize the functions described in the tables of Fig. 3.14.

3.12 Problem: Design a sequential circuit (an electronic lock) that enters an accepting stateonly when it receives some particular four-bit sequence that you specify.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution It is useful to assume that the FSM has a reserved input letter (a “masterkey”) that resets the FSM to the initial state. Design an FSM that has five states,q0, q1, q2, q3, q4. We describe its action on all input letters except for the master key.Advance from qi to qi+1, i = 0, 1, 2, 3, if four successive input letters are correct. Ifany one of them is incorrect, enter the “reject state” q4. Since q3 is the “accept state,”return to it on all inputs. Also return to q4 on all inputs.

3.1 Solutions 41

Adder +1

Compare to p

x7 x6 x5 x4 x3 x2 x1 x0

x7 x6 x5 x4 x3 x2 x1 x0

Figure S.3.2 A modulo-p counter.

3.13 Problem: Design a sequential circuit (a modulo-p counter) that increments a binarynumber by one on each step until it reaches the integer value p, at which point it resetsits value to zero. You should assume that p is not a power of 2.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: We construct such a circuit from a register, an adder and comparator circuit,as suggested by Fig. S.3.2. At each step one is added the contents of the register. Theresult is compared to p and if it equals p in binary, the output of the comparator is setto zero. This is ANDed with the word produced by the adder and sent to the register.Thus, the register is zeroed if one more than its current contents is equal to p.

To determine if the new sum is exactly p, we need only compare the bits of p individ-ually with the bits of the new sum. If they are all equal, the sum has been reached. Acircuit to compare two bits for equality and return 0 if they are not equal is the negationof the EXCLUSIVE OR gate. An AND tree circuit returns a value of 0 if one or more ofthe comparisons returns 0.

3.14 Problem: Give an efficient design of an incrementing/decrementing counter, a se-quential circuit that increments or decrements a binary number modulo 2n. Specifythe machine as an FSM and determine the number of gates in the sequential circuit interms of n.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: An incrementing and decrementing counter can be realized by an adder thateither adds 1 or -1 to the current value. Since the result must be the sum modulo 2n, ifthe sum reaches 2n, it must be reduced to 0. But this is exactly what a traditional adder


does by dropping the most significant bit. To add -1 we take the 2’s complement of 1obtained by complementing every bit and adding 1. This is the all 1’s n-tuple. Thus, ifthe sum is 0 and -1 is added, a traditional adder produces the result which is the all 1’sn-tuple. Since this is the representation for 2n � 1, this is exactly the desired result.

Thus, we can construct an incrementing/decrementing counter modulo 2n from an n-bit register, an adder and gates that add either the n-tuple 00 � � �01 or 11 � � �11 to thecurrent value in the register. The control input can be used as an input to a multiplexerthat selects of these two binary tuples to be input to the adder.

RANDOM-ACCESS MACHINES

3.15 Problem: Given a straight-line program for a Boolean function, describe the stepstaken to compute it during fetch-and-execute cycles of a RAM. Determine whetherjump instructions are necessary to execute such programs.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: See the solution to Problem 2.4(b).

3.16 Problem: Consulting Theorem 3.4.1, determine whether jump instructions are neces-sary for all RAM computations. If not, what advantage accrues to using them?

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Since every T -step RAM computation can be done by a straight-line pro-gram, jump instructions, which are not used by straight-line programs, are not neces-sary. They are very useful, nonetheless, because they can shorten the length of compu-tations.

3.17 Problem: Sketch a RAM program using time and space O(n) that recognizes stringsof the form f0m1m j 1 � m � ng.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Let the RAM CPU have a ternary input alphabet with letters 0, 1 and 2.Let 2 denote the end of the string. Use a register or memory location to count up thenumber of 0’s received as input to the CPU. If the first input is not 0, reject the string.Otherwise, add 1 to the register every time another 0 is received as input. On receivingthe first 1, decrement the value in the register. If no 1’s are received or if the register isnot zero on receiving the input 2, reject the string. Otherwise, accept it.

ASSEMBLY-LANGUAGE PROGRAMMING

3.18 Problem: Write an assembly-language program in the language of Fig. 3.18 to subtracttwo integers.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: To subtract a from b, halt if a is zero and otherwise decrement both a and b.An assembly language program is given below.

3.1 Solutions 43

R0 R0 � R1 Comments

N0 R1 JMP� N1 End if R1 = 0DEC R0 Decrement R0

DEC R1 Decrement R1

JMP+ N0 RepeatN1 CONTINUE

3.19 Problem: The assembly-language instructions of Fig. 3.18 operate on integers. Showthat the operations AND, OR, and NOT can be realized on Boolean variables with theseinstructions. Show also that these operations on vectors can be implemented.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The following programs realizes AND, OR, and NOT under the assumptionthat registers contain the integers 0 or 1. R2 and R3 contain the input data.

R4 R2 AND R3 Comments

R0 0 R0 initialized to 0R1 1 R1 initialized to 1R2 JMP� N1 R4 is 0 if R2 = 0R3 JMP� N1 R4 is 0 if R2 = 0R4 R1 R4 has value 1JMP� N2 Computation complete

N1 R4 R0 R4 has 0N2 CONTINUE

R4 R2 OR R3 Comments

R0 0 R0 initialized to 0R1 1 R1 initialized to 1R4 R0 Initialize R4 to 0R2 JMP� N1 R4 = R3

R4 R1 R4 has value 1JMP� N2 Computation complete

N1 R3 JMP� N2 R4 = R2 = R3 = 0R4 R1 R4 has value 1

N2 CONTINUE

R4 NOT R2 Comments

R0 0 Register R0 initialized to 0R1 1 Register R1 initialized to 1R2 JMP� N1 R4 is 0 if R2 = 1R4 R0 R4 has value 0JMP� N2 Computation complete

N1 R4 R1 R4 has value 1N2 CONTINUE

Vector AND, OR and NOT can be performed on Boolean vectors using the above pro-gram fragments if we represent the vectors as binary numbers and then encode them asintegers. The next bit in a Boolean result vector can be inserted by shifting the current


vector left and then adding the new bit to it with a conditional increment operation.Shifting a vector left one place can be done by adding the vector to itself which is equiv-alent to multiplying it by 2. The next bit in a Boolean vector can be obtained as theremainder in the division of the vector by 2. Division of one number by another canbe done by subtracting the second as many times as necessary so that the remainder iseither 0 or 1, adding 1 to a register each time to produce the quotient. Subtraction of bfrom a can be done by taking the 2’s complement representation of b and adding it toa. Testing for a remainder of 0 or 1 can be done by testing for 0 or decrementing by 1and then testing for 0.

3.20 Problem: Write an assembly-language program in the language of Fig. 3.18 to form xy

for integers x and y.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: An algorithm for this problem starts with temporary result of x, decrementsy and, if not zero, multiplies the temporary result by x. Thus, we put x into register R0

and R1 and the value y into register R2. We decrement R2, and if not zero, multiplyR1 times R0 and leave the result in R0. To multiply the contents of R0 and R1, we copyR0 into R4 and R1 into R5. We then decrement R5 and, if the result is not zero, addR4 to R0. To add R4 to R0 we save R4 in R6 and increment R0 once and decrement R6

once until R6 is 0.

R0 R0 + R1 Comments

R0 x R0 contains x initiallyR1 x R0 contains x initiallyR2 y R2 contains y initially

N0 DEC R1 Decrement R1

R1 JMP� N5 Exit if the last value of R1 is 1R4 R0 Start multiplication by saving R0

R5 R1 Save R1

DEC R5 Decrement R5

R5 JMP� N5 Exit if R5 is 0R6 R4 Copy R4 to start multiplication by addition

N1 R6 JMP� N1 End if R6 = 0INC R0 Increment R0

DEC R6 Decrement R6

JMP+ N1 Repeat incrementing and decrementingN2 CONTINUE

JMP+ N0 Repeat multiplication by xN5 CONTINUE

3.21 Problem: Show that the assembly-language instructions CLR Ri, Ri Rj , JMP+ Ni,and JMP� Ni can be realized from the assembly-language instructions INC, DEC,CONTINUE, Rj JMP+ Ni, and Rj JMP� Ni.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The program for CLR Ri is straightforward and given below. The one forRi Rj clears registers Ri and Rk and then procedes to increment both of them while

3.1 Solutions 45

decreasing Rj to 0 which results in copying Rj into both Ri and Rk. It then restoresRj to its original value by copying Rk into Rj.

CLR Ri Comments

N0 Ri JMP� N1 If Ri = 0, exitDEC Ri Decrement Ri

0 JMP+ N0 Check if Ri is zeroN1 CONTINUE

Ri Rj Comments

CLR Ri Invoke routine for CLRCLR Rk Invoke routine for CLR

N2 Rj JMP� N4 Done if Rj = 0INC Ri Increment Ri

INC Rk Increment Rk

DEC Rj Decrement Rj

Rj JMP� N3 If Rj is zero, restor its value0 JMP+ N2 Rj is not yet zero; continue copying process

N3 INC Rj Increment Rj

DEC Rk Decrement Rk

Rk JMP+ N4 Done if Rk is 00 JMP� N3 Resume copying Rk into Rj

N4 CONTINUE

JMP� Ni Comments

CLR Rk Invoke routine for CLRRk JMP� Ni Jump is unconditional

JMP+ Ni Comments

CLR Rk Invoke routine for CLRRk JMP+ Ni Jump is unconditional

TURING MACHINES

3.22 Problem: In a standard Turing machine the tape unit has a left end but extends indef-initely to the right. Show that allowing the tape unit to be infinite in both directionsdoes not add power to the Turing machine.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: To answer this question we show that for every two way infinite TM M1 =(�1,Q1, Æ1, s1,h1) there is an equivalent standard TM M2 = (�2,Q2, Æ2, s2,h2). M2

has its tape split into an upper and lower track so each tape cell can hold two symbols.On the leftmost position we place a special end of tape marker .. We start simulatingM1 on the upper track of the tape of M2. When there is a transition that will move thehead to the left of the marker . we change tracks and start simulating M1 on the lowertrack of M2.The tape of M2 is just a folded version of the tape of M1. The alphabet �2 is (�1 ��1) [ f.g [ �. For each transition Æ1(q,x) = (r, y), where x, y 2 Q1, x 2 �1,


y 2 �1 [ fL,Rg we construct two new transitions Æ2 and Æ02 that correspond to thesimulation of Æ1 in the upper and lower track respectively. Æ2 and Æ02 read and write pairsof symbols and the head movements of Æ02 are inverted. M2 has also two transitions forchanging tracks when the head reaches the symbol ..

3.23 Problem: Describe in detail a Turing machine with unlimited storage capacity thatrecognizes the language f0m1mj1 � mg.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Start with the machine in state qstart with the head positioned in the left-most cell of the tape. If the symbol under the head is different from 0 then go to stateqreject. If it is a 0 then replace it with � (blank symbol), go to state qfind1 and movethe head one cell to the right.

In state qfind1 move the head to the right until it finds a 1. In this case go to stateqskip1 and move the head one cell to the right. If it finds a � while in state qfind1 thengo to state qreject.

In state qskip1 move the head to the right until it reads the first blank symbol. Thenmove the head one cell to the left, replace the 1 with � and go to state qfind0. If whilein state qskip1 if encounters a 0 then go to state qreject.

In state qfind0 move the head to the left until it finds a 0 and then go to state qskip0.If it reads a � before a 0 check if the symbol to the right is also a �. If so go to stateqaccept; otherwise go to state qreject (there are more 1s than 0s in the input).

In state qskip0 move the head to the left until it finds the first �. Then move the headto the right one cell, replace the 0 with a �, go to state qfind1.

3.24 Problem: Sketch a proof that in O(n4) steps a Turing machine can verify that a par-ticular tour of n cities in an instance of the Traveling Salesperson Problem satisfies therequirement that the total distance traveled is less than or equal to the limit k set onthis instance of the Traveling Salesperson Problem.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: For concreteness, let an instance of TSP be represented by a tuple of theform (n; d1,2, d1,3, : : : , d1,n; d2,3, : : : , d2,n; : : : ; dn�1,n,D). Such a string is in thelanguage TSP if there is an ordering of the n cities such that the length of the tour isat most D. Imagine writing this string on a three-track tape. Let the jth semicolon onthe top track be associated with the jth city. Thus, the distances to the right of the jthsemicolon are the distances from the jth city to all higher numbered cities. Also, letthe ith comma following the j semicolon be associated with the city whose number isi more than j. Let l be the length of the instance of TSP. It follows that l = O(n2).

Assume that such a string is in TSP and that a choice agent writes on the second trackbelow the jth semicolon an integer i such that the tour passes through the jth city byentering via edge (j, j + i) or edge (j + i, j). Moving generally left to right, advancea marker on the third tape from the jth semicolon to the (i � 1)st comma to its right,thereby identifying the length of an edge in the tour. This process takes time O(n2)because O(n) steps are needed to move each marker from a semicolon to a comma.Thus, the time is O(l), linear in the length of the input string.

Now using the third track to mark cells, copy each of the tour edge lengths to the blankportion to the right of the string and add them up. Moving distances takes time O(l2)

3.1 Solutions 47

because there are at most l characters in all n distances and at most l cells must becrossed to move each one of them. The addition of two numbers placed on contiguoustape cells can be done in time proportional to the square of their length with their sumplaced in the cells occupied by the number to the right. Thus, the addition can be donein time O(

Pni=1 l

2i ) where the lengths l1, l2, : : : , ln sum to at most l. Thus, this time

bound is O(l2).

Finally, compare the result to D by bouncing back and forth between the string repre-senting D and the sum. Since the portion of the occupied tape has length O(l), thistakes time O(l2) which is O(n4).

3.25 Problem: Design the additional circuitry needed to transform a sequential circuit fora random-access memory into one for a tape memory. Give upper bounds on the sizeand depth of the next-state and output functions that are simultaneously achievable.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Since the head can only move from one cell to at most a cell with a numberthat is one higher or lower than the current number, we modify the random-accessmemory by adding circuits to add �1, 0, or 1 to the current value of the memoryaddress register. This adds at most O(b) gates to the circuit size and at most O(log b)to the depth of the next-state/output functions for the random-access memory unit.

3.26 Problem: In the proof of Theorem 3.8.1 it is assumed that the words and their ad-dresses in a RAM memory unit are placed on the tape of a Turing machine in order ofincreasing addresses, as suggested by Fig. 3.40 (which is repeated as Fig. S.3.3). The ad-dresses, which are dlogme bits in length, are organized as a collection of ddlogme=beb-bit words. (In the example, b = 1.) An address is written on tape cells that imme-diately precede the value of the corresponding RAM word. A RAM address addr isstored on the tape to the left in the shaded region.

Assume that markers can be placed on cells. (This amounts to enlarging the tape al-phabet by a constant factor.) Show that markers can be used to move from the firstword whose RAM address matches the ib most significant bits of the address a to thenext one that matches the (i + 1)b most significant bits. Show that this procedure canbe used to find the RAM word whose address matches addr in O((m=b)(logm)2)Turing machine steps by a machine that can store in its control unit only one b-bitsubword of addr.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Assume that the addresses placed on the tape appear in increasing orderfrom left to right. Assume that the first b-bit word in an address is the most significantword and that successive such address words have decreasing significance. Also, assume

} � } �}

w

1

w

2

�

0 0 w

0

0 1 1 0 101 1

}

w

3

�

Figure S.3.3 A TM tape with markers on words and the first bit of each address.


that the first and most significant word in each address is marked (as indicated by thediamonds in Fig. S.3.3) as are the data words (as indicated by the spades in Fig. S.3.3).Let the address addr be specified as q = ddlogme=be b-bit words stored in decreasingsignificance from left to right in the first q tape cells and let its most significant wordbe marked.The TM reads the currently marked word, c, of addr and moves right to the firstmarked address word associated with data whose value agrees with c, erasing all previousmarks as it advances. This process places the head over the first cell whose value agreeswith c. If the next cell does not contain a RAM data word, it now travels to the rightend of the non-blank portion of the tape advancing this and subsequent address markson address words one cell to the right. This insures that on the next pass it will comparethe word of next lower significance in addr with the appropriate word in the simulatedRAM. This process repeats until all words of addr have been marked in which case theleftmost mark on an address word is that of the RAM word whose address is addr.The random-access memory has m b-bit words and a tape cell can hold a b-bit wordas well as a mark. Each address occupies q tape cells whereas the corresponding dataoccupies one cell. Thus, there are O(mq) cells used to store addresses and data forentries in the random-access memory. Thus, one pass over the tape can be done inO(mq) steps. The number of passes over the tape is equal to the number of words inone address, which is q. It follows that the data whose address matches addr can befound in O(mq2) steps. But O(mq2) = O(m(logm=b)2), which is smaller than thebound requested.

3.27 Problem: Extend Problem 3.26 by demonstrating that the simulation can be done witha binary tape symbol alphabet.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: This problem can be addressed as follows: Given an arbitrary tape alphabet,find a binary encoding of its letters. Extend the tape alphabet to include the specialsymbols such as semicolon and comma as well as the marked versions of these symbols.Now choose any encoding of this alphabet that assigns a fixed number of bits to everyletter. The control unit is then designed to read this fixed number of bits before itsimulates the state transition that would be performed by the original machine. Aftermaking this transition, it procedes to replace the fixed-length bit string on its tape withthe encoding of the new symbol that is written by the original Turing machine. Also, ifnecessary, it moves the head by the number of binary tape cells needed to place it overthe first cell corresponding to the original head placement.

3.28 Problem: Extend Theorem 3.8.1 to show that there exists a Turing machine that cansimulate an unbounded-memory RAM.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: When the RAM is simulated an address may be produced that is largerthan an existing address. This requires the creation of new address/data pairs for thehigher indexed RAM locations as well as expanding the address portions of existingaddress/data pairs. The existing address/data pairs must have inserted as their newmost significant b-bit words the value 0.The address/data pairs for the enlarged set of address can be created as follows: a) shiftthe existing address/data pairs words right to create space for the new most significant

3.1 Solutions 49

b-bit words each of word (a procedure for this purpose is sketched below) and insert theword 0 in the most significant place in each of these address words; b) for each of the2b b-bit words except the first, copy the newly expanded originally address/data pairsand replace their most significant b-bit words by the appropriate value and replace theirdata words by 0.

A Turing machine that shifts the contents of contiguous tape cells to the right by oneplace is easily realized by placing marks on the cell to the left of the leftmost characterto be shifted as well as the rightmost character to be shifted. It then moves to the rightmarker, copies the value under the head to the cell to the right. If the cell to the left ofthe marked cell contains the left marker, the process is complete; otherwise, the processcontinues at the right marker.

3.29 Problem: Sketch a proof that every Turing machine can be simulated by a RAM pro-gram of the kind described in Section 3.4.3.

Hint: Because such RAM programs can only have a finite number of registers, encodethe contents of the TM tape as a number to be stored in one register.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: We must show that the control and tape units can be simulated. We use afinite number of RAM registers for this purpose. While this may not seem possible, wenote that no restriction is placed a priori on the size of the largest integer that can beheld in a RAM register. As shown below, we use a finite number of registers to recordthe state transitions and two registers to record the contents of the tape.

Because a control unit is an FSM defined by a next-state/output function, we need onlydescribe a procedure to compute any fixed function with a RAM program. We reasonedin Problem 3.27 that tape alphabet can be binary. We make this assumption here.

To simulate an FSM with binary input alphabet, let the FSM state be represented byan integer in register R0. Let the input and output be located in registers R1 and R2.Form a state/input pair by shifting R0 left one place (add it to itself ) and adding R1 toit.

Let q be the number of state/input pairs of the FSM (values in R0 after the aboveoperation). These pairs are associated with the integers 0, 1, 2, : : : , q � 1. To computethe next state of the FSM, we write a RAM program to jump to location Nk when R0

contains k. In this location, we place an instruction of the form R0 r which causesr to be the next state. We follow this by an instruction of the form R2 s whichproduces the output s. The next instruction jumps back to the first instruction of theprogram.

To cause the RAM program to jump to location k when R0 contains k, we use thefollowing code fragment:


R0 JMP� N0 Go to N0 if R0 = 0DEC R0 Decrement R0

R0 JMP� N1 Go to N1 if R0 = 1DEC R0 Decrement R0

R0 JMP� N2 Go to N2 if R0 = 2:::

DEC Rq�1 Decrement R0

R0 JMP� Nq�1 Go to Nq�1 if R0 = q � 1We simulate a binary tape unit by two integers, one of which represents the binarysequence to the left of the tape head and the other which represents the binary sequenceto its right. In the solution to Problem 3.28 we give a procedure to shift a binarysequence left one place, adding in a new bit as does. Also, we give a procedure to shift abinary sequence right one place, reading the rightmost bit that is produced by this shift.We use these procedures to simulate the movement of the head on the tape. Togetherwith the description of the control unit, we have a program that simulates an arbitraryTuring machine.

COMPUTATIONAL INEQUALITIES FOR TURING MACHINES

3.30 Problem: Show that a one-tape Turing machine needs time exponential in n to com-pute most Boolean functions f : Bn 7! B on n variables, regardless of how muchmemory is allocated to the computation.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: This follows from Corollary 3.9.1 and Theorem 2.12.1. Corollary 3.9.1shows that the time T compute a Boolean function f with circuit size C(f) is at least(pC(f)) while Theorem 2.12.1 shows that C(f) is at least 2n

n(1� �)� 2n2 for

most Boolean functions on n variables.

3.31 Problem: Apply Theorem 3.2.2 to the one-tape Turing machine that executes T steps.Determine whether the resulting inequalities are weaker or stronger than those given inTheorem 3.9.2.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: We derive an upper bound on the number of states which a TM can assumethat executes T steps and uses m cells. We then apply Theorem 3.2.2.

The number of distinguishable states of the tape unit is at most the product of thenumber of different positions for the tape head (m) with the sum of the maximumnumber of different values that can be assumed by the first j cells of the tape for 1 �j � m (a blank can reside to the right of the jth cell). Since each cell can have at mostj�j values, the tape can assume at most the following number of values.

mXj=1

j�jj = �j�jm+1 � j�j�=(j�j � 1)

It follows that the T -step TM can have at most�mj�jm+2

� jQj states where jQj is thenumber of states of the control unit. Let a � max(j�j, jQj). Then, the TM has atmost mam+2 states.

3.1 Solutions 51

From Theorem 3.2.2 and the fact that two n � n matrices can be multiplied withO(n3) operations, since n = mam+2 in this case, the following bounds hold:

C(f) = O(m3a3(m+2)T )

D(f) = O(m logT )

The first inequality given above, which is of the form O(S3bST ), is weaker than thefirst inequality given in Theorem 3.9.2, whereas the second given above, which is of theform O(S logT ), is stronger than the second in Theorem 3.9.2 when S logT is muchsmaller than T logS.

3.32 Problem: Write a program in your favorite language for the procedure WRITE OR(t,m) introduced in Fig. 3.27.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: For the tth step, 1 � t � T , this program writes a straight-line program forthe vector OR of vj,t, 0 � j � m � 1, where j indexes a memory cell and vj,t is abinary b-tuple. Let vj,t,r be the rth bit, 1 � r � b, in the jth cell on the tth step.Then, for each value of j and t the OR must be taken of vj,t,r. This can be done bytaking the OR of vj,t,1 and vj,t,2, taking the OR of the result with vj,t,3, etc.

3.33 Problem: Write a program in your favorite language for the procedure WRITE CELL

CIRCUIT(t, m) introduced in Fig. 3.27.Hint: See Problem 2.4.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: This program is straightforward.

FIRST P-COMPLETE AND NP-COMPLETE PROBLEMS

3.34 Problem: Show that the language MONOTONE CIRCUIT VALUE defined below isP-complete.

MONOTONE CIRCUIT VALUE

Instance: A description for a monotone circuit with fixed values for its input variablesand a designated output gate.Answer: “Yes” if the output of the circuit has value 1.

Hint: Using dual-rail logic, find a way to translate (reduce) a string in the languageCIRCUIT VALUE to a string in MONOTONE CIRCUIT VALUE by converting in loga-rithmic space (in the length of the string) a circuit over the standard basis to a circuitover the monotone basis. Note that, as stated in the text, the composition of twologarithmic-space reductions is a logarithmic-space reduction. To simplify the con-version from non-monotone circuits to monotone circuits, use even integers to indexvertices in the non-monotone circuits so that both even and odd integers can be usedin the monotone case.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Dual-rail logic circuits are described in Problem 2.12. A standard circuit isconverted to a dual-rail one by replacing each wire by a pair of wires carrying a value and


its complement. Let steps in the original straight-line program be given even numbers.If step 2j carried the value x, in the new program step 2j carries the value x and step2j + 1 carries the value x.

As a consequence of this representation, input step (2k read 2i) is replaced by the steps(2k read 2i) and (2k+ 1 read 2i+ 1) where the input 2i and 2i+ 1 are complements.Each NOT gate is replaced by a twisted pair of wires, and AND and OR gates are eachreplaced by one AND and one OR gate. Step (2k NOT 2i) in the original program isreplaced by steps (2k = 2i + 1) and (2k + 1 = 2i). (The equality operator can berealized by either AND or OR with two identical arguments.) This assignment realizesthe DRL-NOT as a pair of twisted wires. Step (2k AND 2i 2j) is replaced by the steps(2k AND 2i 2j) and (2k+ 1 OR 2i+ 1 2j+ 1). Similarly, (2k OR 2i 2j) is replaced bythe steps (2k OR 2i 2j) and (2k + 1 AND 2i+ 1 2j + 1).

Given a straight-line program, the generation of the new steps from the original stepscan be done by examining one step at a time and performing computations to incre-ment variables. This represents an amount of space that proportional to the length ofthe numbers involved. Since these numbers refer to steps in the program, they havea length logarithmic in the length of the program. It follows that the translation canbe done deterministicaly in logarithmic working space. Because a straight-line pro-gram evaluates to 1 if and only if its monotone equivalent evaluates to 1, a straight-lineprogram is in CIRCUIT VALUE if and only if the corresponding dual-rail straight-lineprogram is in MONOTONE CIRCUIT VALUE. Since every language in P can be re-duced to CIRCUIT VALUE in logspace and the composition of two logspace reductionsis a logspace reductions, every language in P can be reduced to MONOTONE CIRCUIT

VALUE in logspace. Thus, MONOTONE CIRCUIT VALUE is P-complete.

3.35 Problem: Show that the language FAN-OUT 2 CIRCUIT SAT defined below is NP-complete.

FAN-OUT 2 CIRCUIT SAT Instance: A description for a circuit of fan-out 2 with fixedvalues for its input variables and a designated output gate.Answer: “Yes” if the output of the circuit has value 1.

Hint: To reduce the fan-out of a vertex, replace the direct connections between a gateand its successors by a binary tree whose vertices are AND gates with their inputs con-nected together. Show that, for each gate of fan-out more than two, such trees can begenerated by a program that runs in polynomial time.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Since it is desirable but not necessary that a step in a straight-line programoperate on the results of previous steps, we simplify the design of a translator by drop-ping this restriction.

Since a polynomial-time RAM program runs in polynomial time on a Turing machine,we describe a RAM translator program. This program visits every step of the straight-line program and collects the largest integer, L, used to identify a step. It then adds afield to each step that points to the root of a binary tree. For example, if (nOP p q) isa step, a field is added immediately after n that is initially � to form (n � OPp q).

The program revisits each step of the straight-line program for the purpose of reducingthe fan-out. Consider the first step (the sth) that contains a reference to another step.

3.1 Solutions 53

If step s refers to step n, increment L, replace � in step n with the new value of L,replace n in step s by L, and create the step (L� = n), a new step that says the valueof step L is the value of step n. (The equality operator can be realized by either AND

or OR with two identical arguments.)

This additional step does not change the fan-out of step n but it does cause step s torefer to this new step which has fan-out one. When a step is encountered that refersto a step n for which the new field is not � but is some integer, say r, the programincrements L, visits step r, and if its second field is not � but is r1 visits step r1 and itspossible successors until a step is found whose second component does contain �. Atthis point the program replaces� by L and creates a new record of the form (L� = q)where q is the label of the last step visited. The last operation reduces the fan-out ofstep n by one and increases the fan-out of step q from 1 to 2. Following in this fashion,the fan-out of every step is reduced to 2.

Let fi be the fan-out of step i. Because a constant number of steps is performed for thefirst step pointing to step i, andO(t) steps on the tth step to point to step i, the numberof operations performed, T , is proportional to S =

Pki=1 fi + (fi � 1) + � � � + 1,

where k is the number of different steps in the original program. Here t � fi andN =

Pki=1 fi both the number of times that some step is used by another step and the

number of times labels appear in steps. Thus, N is at most twice the number of stepsand T is at most the square of the number of steps.

3.36 Problem: Show that clauses given in the proof of Theorem 3.9.7 are satisfied onlywhen their variables have values consistent with the definition of the gate type.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Let gi be the value of the result of step i. The function associated with(i READ x) is gi = x. But the clauses (gi _ x) and (gi _ x) are both 1 when bothgi and x are 1 or both gi and x are 1. These are exactly the conditions under whichgi = x.

The function associated with (i NOT j) is gi = gj . But the clauses (gi _ gj) (gi _ gj)are both 1 when gi is 1 and gj is 1 (that is, gj is 0), or gi is 0 and gj is 1. These areexactly the conditions under which gi = gj.

The function associated with (i OR j k) is gi = gj _ jk. But the clauses (gi _ gj),(gi _ gk), and (gi _ gj _ gk) are all True when either gi is 1 and one of gj and gk isTrue or when gi is 0 and both gj and gk are 1. These are exactly the conditions underwhich gi = gj _ jk.

The function associated with (i AND j k) is gi = gj _ jk. But the clauses (gi _ gj),(gi _ gk), and (gi _ gj _ gk) are all True when gi is 1 and both gj and gk are True orwhen gi is 0 and either gj or gk are False.

The function associated with (i OUTPUT j) is 1 exactly when gj is 1 which is exactlythe condition under which the clause (gj) is True.

3.37 Problem: A circuit with n input variables fx1,x2, : : : ,xng is satisfiable if there isan assignment of values to the variables such that the output of the circuit has value1. Assume that the circuit has only one output and the gates are over the basis =fAND, OR, NOTg.


a) Describe a nondeterministic procedure that accepts as input the description of acircuit in POSE and returns 1 if the circuit is satisfiable and 0 otherwise.

b) Describe a deterministic procedure that accepts as input the description of a circuitin POSE and returns 1 if the circuit is satisfiable and 0 otherwise. What is therunning time of this procedure when implemented on the RAM?

c) Describe an efficient (polynomial-time) deterministic procedure that accepts as in-put the description of a circuit in SOPE and returns 1 if the circuit is satisfiableand 0 otherwise.

d) By using Boolean algebra, we can convert a circuit from POSE to SOPE. We canthen use the result of the previous question to determine if the circuit is satisfiable.What is the drawback of this approach?

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: a) A non-deterministic procedure for determining if a circuit in POSE issatisfiable chooses the values for x1,x2, : : : ,xn for which the circuit has output 1 andthen verifies that this assignment is correct by computing the values at the gates.b) A deterministic procedure for determining if a circuit in POSE is satisfiable woulditerate through all the possible values of the inputs x1,x2, : : : ,xn. As there are 2n

possible assignments this procedure will run in exponential time.

c) A circuit of SOPE is satisfiable if it has a product not containing both an inputvariable and its complement. This can be determined by scanning the description ofthe circuit. This procedure will run in polynomial time.d) A procedure for converting from a circuit in POSE to a circuit in SOPE wouldnormally require exponential time. If this procedure would take polynomial time thenthere will be a deterministic polynomial time procedure for determining if a circuit inPOSE is satisfiable and we would have a polynomial-time solution to P = NP .

CENTRAL PROCESSING UNIT

3.38 Problem: Write an assembly-language program to multiply two binary numbers usingthe simple CPU of Section 3.10. How large are the integers that can be multipliedwithout producing numbers that are too large to be recorded in registers?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Since the product of two n-bit binary numbers is a 2n-bit number and oursimple CPU can hold 16-bit numbers, we limit the multiplication program to 8-bitnumbers.The program of Figure S.3.4 uses the algorithm of Section 2.9; that is, it multiplies vby u by adding v2j to a running sum if uj = 1. We form v2j by shifting v left onsuccessive cycles using CIL. Determination of the values of uj, 0 � j � 7 can be donein a variety of ways. We choose to form the AND of u with 8 different Boolean vectorscontaining a single 1 and then use JZ to jump to determine whether to add the currentshifted version of v.Location ADDR 8 holds v and shifted versions thereof. Location ADDR 9 holds u.The partial sum is stored in location ADDR 10. The 8 Boolean vectors containinga single 1 in the 8 least significant places are stored in locations ADDR 0, ADDR 1,: : : , ADDR 7. In this program we use indirect addressing and use location ADDR 11

3.1 Solutions 55

ORG 0 Program is stored at location 0.

CLA Clear AC

STA ADDR 10 Initialize partial sum to 0

N 0 LDA ADDR 9 Load AC with u

AND I ADDR 11 Expose bit 0 of u

JZ N 1 If AC = 0, skip adding v20

LDA ADDR 10 Load current partial sum

ADD ADDR 8 Compute new partial sum

STA ADDR 10 Store new partial sum

CLA Clear AC

ADD ADDR 8 Load v2j

CIL Shift v2j left

STA ADDR 8 Store v2j+1

N 1 LDA ADDR 11 Load address for indirection

ADD ADDR 0 Increment address for indirection

STA ADDR 11 Store new address for indirection

ADD ADDR 12 Add 2’s complement of 7

JZ N 0 Repeat the addition if not done

END

Figure S.3.4 A program to add two binary 8-bit numbers.

for the address at which to find the address of the next Boolean vector to AND withu. Indirection can be avoided by replicating code. The 2’s complement of 7, thehighest addressed used for a Boolean vector, is stored in location ADDR 12. This valueis added to the address used for indirection. When the sum is zero, this address hasreached the value 7.

3.39 Problem: Assume that the simple CPU of Section 3.10 is modified to address an un-limited number of memory locations. Show that it can realize any Boolean function bydemonstrating that it can compute the Boolean operations AND, OR, and NOT.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: We slightly modify the programs given in the solutions to Problem 3.19 toaccount for the differences in syntax between RAM assembly language statements andthe simple CPU assembly language statements.

3.40 Problem: Design a circuit to produce the timing variables tj, 1 � j � k, of the simpleCPU. They must have the property that exactly one of them has value 1 at a time andthey successively become 1.


Hint: Design a circuit that counts sequentially modulo k, an integer. That is, it incre-ments a binary number until it reaches k, after which it resets the number to zero. SeeProblem 3.13.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The solution to Problem 3.13 describes a circuit to realize a counter modulo-p. We use such a counter to count to modulo-k and supply the output to a decoderwith at least k outputs. The first k outputs are the desired values.

3.41 Problem: Complete the design of the CPU of Section 3.10 by describing circuits forPC, MAR, MDR, OPC, INR, and OUTR.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Fig. 3.38 shows the microinstructions affecting each register. Those affect-ing AC are used to construct the circuit shown in Figure 3.39. The control variablesaffecting the registers PC, MAR, MDR, OPC, INR, and OUTR are described on page144.When L(PC, PC+1) = 1, the value of PC is incremented and when L(PC, MDR) =1, PC is given the value of MDR. Thus, the input to the register PC is a multiplexerone input of which is MDR and the other is an adder that has as one input the currentvalue of PC and as a second the value 1.When L(MAR, PC) = 1, MAR is given the value of PC and when L(MAR, MDR) =1 it is given the value of MDR. Thus, MAR has as input a multiplexer whose two inputsare PC and MDR.When L(MDR, M) = 1, MDR is given the value M read from memory and whenL(MDR, AC) = 1, MDR is given the value of AC. Thus, MDR has as input a multi-plexer whose two inputs are MDR and AC.When L(OPC, MDR) = 1, OPC is given the value of MDR. Thus, the OPC has asinput the value of MDR each bit of which is ANDed with L(OPC, MDR).When L(OUTR, AC) = 1, OUTR is given the value of AC. Thus, the OUTR has asinput the value of AC each bit of which is ANDed with L(OUTR, AC).The register INR always supplies its input to the multiplexer feeding AC.

3.42 Problem: Show that an indirect store operation can be simulated by the computer ofSection 3.10.Hint: Construct a program that temporarily moves the value of AC aside, fetches theaddress containing the destination for the store, and uses Boolean operations to modifya STA instruction in the program so that it contains the destination address.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The code shown in Fig. S.3.5 is self-modifying. It simulates STA I A 1,indirect addressing through address A 1, by moving the contents of AC to locationA 0, loading AC with A 1 which contains the new address A NEW in its twelve leastsignificant positions, and ANDs this word with the bit pattern 0110(1)16 where (0110)is the binary encoding for STA. The result is the bit pattern for STA A NEW. Thisvalue is stored in location Y (replacing NOOP) which immediately follows the nextinstruction, LDA A 0, which restores the value of AC. Since the instruction that isexecuted after LDA A 0 is the new instruction STA A NEW, this piece of assemblylanguage code realizes STA I A 1 through self modification.

3.1 Solutions 57

ORG X Program is stored at location X.

STA A 0 Store AC temporarily.

LDA A 1 Load address A NEW on which to indirect.

AND A 2 A 2 contains (STA)11...1.

STA Y Store new instruction, namely, STA A NEW.

LDA A 0 Restore AC to original value.

Y NOOP Program executes STA with address A NEW.

END

Figure S.3.5 A program to realize indirect addressing with STA.

3.43 Problem: Write an assembly-language program that repeatedly examines the inputregister until it is nonzero and then moves its contents to the accumulator.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: This problem has a simple solution. Repeatedly jump back to the instruction(IN) that loads AC until AC is non-zero.

3.44 Problem: Sketch an assembly-language program to emulate a target CPU by a hostCPU under the assumption that each CPU’s instruction set supports indirection. Pro-vide a skeleton program that reads an instruction from the target instruction set anddecides which host instruction to execute. Also sketch the particular host instructionsneeded to emulate a target add instruction and a target jump-on-zero instruction.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: The general idea here is that after reading an instruction from a program forthe target CPU, the host CPU branches to a small set of assembly language instructionsto emulate the target CPU instruction. Let each target CPU instruction contain threefields, a bit pattern for an instruction name, an address, and a bit denoting whether ornot addressing is indirect.The target and emulator programs are located in separate portions of memory. Theemulator program reserves a location for the address of the current instruction of thetarget program. The emulator moves the current values of its registers aside to reservedlocations. Using indirection, it loads the first target instruction into the accumulator.It then stores the accumulator in a reserved location so that it can perform multiple

ORG X Program is stored at location X.

Y IN Load AC with the value of IN.

JZ Y Continue unless AC is zero.

END

Figure S.3.6 A program that loads AC with the value of IN when IN becomes non-zero.


operations on it after which it increments the address of the current instruction of thetarget program. It then begins to emulate the current target instruction.The emulator loads the target instruction three times, ANDing it with bit vectors to de-compose into its three parts which it stores in three memory locations. (If host addressesare stored in different bit locations than are target addresses, some address manipula-tion will be necessary.) It then tests the instruction bit pattern against the possible targetinstruction bit patterns and executes the comparable host instruction with a direct orindirect address depending on the value of the indirect bit. This comparison processcould be done by successively substracting each of the possible instruction bit patternsuntil one is found that causes the accumulator to be zero.If the target instruction is a direct add, the host reloads all registers and then executes itsadd instruction indirectly through the address of the location that contains the addressportion of the target instruction. If the target instruction uses indirection, an indirectload through the register containing the address portion of the target instruction isneeded to load the address at which to execute the instruction. The steps for a directadd are then followed. Similar steps are followed for a jump-on-zero target instruction.

C H A P T E R

Finite-State Machines andPushdown Automata

4.1 SolutionsFSM MODELS

4.1 Problem: Let M = (�,,Q, Æ,�, s,F ) be the FSM model described in Defini-tion 3.1.1. It differs from the FSM model of Section 4.1 in that its output alphabet has been explicitly identified. Let this machine recognize the language L(M ) consist-ing of input stringsw that cause the last output produced by M to be the first letter in. Show that every language recognized under this definition is a language recognizedaccording to the “final-state definition” in Definition 4.1.1 and vice versa.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Given an FSM M described in Definition 3.1.1 that recognizes the languageL of strings that cause M to produce as its last output the first letter a in , make thestates inM that produce such letters final states. This “new” version of M is an instanceof an FSM given by Definition 4.1.1 that recognizes L. Similarly, given an FSM givenby Definition 4.1.1 that recognizesL, change its definition so that it outputs letter a ineach state that is designated as final. This “new” machine conforms to Definition 3.1.1and recognizes L.

4.2 Problem: The Mealy machine is a seven-tuple M = (�,,Q, Æ,�, s,F ) identicalin its definition with the Moore machine of Definition 3.1.1 except that its outputfunction � : Q� � 7! depends on both the current state and input letter, whereasthe output function � : Q 7! of the Moore FSM depends only on the current

59

60 Chapter 4 Finite-State Machines and Pushdown Automata

state. Show that the two machines recognize the same languages and compute the samefunctions with the exception of �.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The solution to this problem is the same as solution to Problem 3.6.

4.3 Problem: Suppose that an FSM is allowed to make state �-transitions, that is, statetransitions on the empty string. Show that the new machine model is no more powerfulthan the Moore machine model.

Hint: Show how �-transitions can be removed, perhaps by making the resultant FSMnondeterministic.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Let state q1 have an �-transition to state q2 and suppose that there is a tran-sition into state q1 on input a (which itself could be �) from q0. Observe that thetransition from q1 to q2 on input � can be replaced by a transition from q0 to q2 oninput a� = a without changing the set of strings that are accepted by the FSM. Eachapplication of this procedure reduces the number of �-transitions without changing thelanguage recognized. Applying this procedure as many times are there are �-transitionsconverts the FSM to one without any �-transitions recognizing the same language.

EQUIVALENCE OF DFSMS AND NFSMS

4.4 Problem: Functions computed by FSMs are described in Definition 3.1.1. Can aconsistent definition of function computation by NFSMs be given? If not, why not?

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: A function must have one value for each point in its domain. Given anNFSM there may be many different output sequences for each input sequence as thechoice agent ranges over all of its corresponding sequences. It is not clear how to choosewhich value to assign to each input sequence. In the case of language recognition byan NFSM, our goal is to get the machine to enter a final state or to produce a specificvalue, say 1. In this case the issue is whether a choice input can produce this result, forwhich there is no ambiguity; either it can or it cannot. Hence, language recognitioncan be defined unambiguously for NFSMs even if function computation cannot.

4.5 Problem: Construct a deterministic FSM equivalent to the nondeterministic FSMshown in Fig. 4.32 (repeated here as Fig. S.4.1).

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The next-state tables for the equivalent machine, Mequiv, is given in Fig-ure S.4.2. All states are final states except for fq0g and fq0, q2g.

REGULAR EXPRESSIONS

4.6 Problem: Show that the regular expression 0(0�10�)+ defines strings starting with 0and containing at least one 1.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Let r = (0�10�). Then 0r+ denotes the strings beginning with 0 followedby strings denoted by r+ which is r+r2+r3+� � �+rk+� � � . A string in rk contains k

4.1 Solutions 61

0 0, 11

0

0, 1

1

1

Startq0 q1

q2 q3

00

Figure S.4.1 A nondeterministic FSM.

1’s with zero or more 0’s between them. It follows that r+ describes strings containingat least one 1 and 0r+ denotes strings beginning with 0 and containing at least one 1.

4.7 Problem: Show that the regular expressions 0�, 0(0�10�)+, and 1(0 + 1)� partitionthe set of all strings over 0 and 1.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: It follows from Problem 4.6 that 0(0�10�)+ denotes strings beginning with0 and containing at least one 1. Clearly, 1(0+1)� denotes strings beginning with 1 andcontaininng 0’s and 1’s. The only remaining case is that of strings that do not containany 1’s, that is, just 0’s, namely, 0�.

4.8 Problem: Give regular expressions generating the following languages over � = f0, 1g:a) L = fw j w has length at least 3 and its third symbol is a 0gb) L = fw j w begins with a 1 and ends with a 0gc) L = fw j w contains at least three 1sg. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) (0 + 1)20(0, 1)�

b) 1(0 + 1)�0c) (0�10�)3(0 + 1)�

4.9 Problem: Give regular expressions generating the following languages over � = f0, 1g:a) L = fw j w is any string except 11 and 111gb) L = fw j every odd position of w is a 1g. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) To construct the complement of a set, construct an FSM that accepts the set andtake the union of the regular expressions associated with the non-accepting states.An FSM accepting 11+ 111 has states q1, q2, q3, q4, and q5 where q1 is the initial


qequiv a ÆMequiv(qequiv, a)

fq0g 0 fq0, q2gfq0g 1 fq1, q2gfq0, q2g 0 fq0, q2, q3gfq0, q2g 1 fq1, q2gfq1, q2g 0 fq0, q1, q3gfq1, q2g 1 fq0, q1gfq0, q1g 0 fq0, q1, q2gfq0, q1g 1 fq0, q1, q2gfq0, q2, q3g 0 fq0, q1, q2, q3gfq0, q2, q3g 1 fq1, q2gfq0, q1, q3g 0 fq0, q1, q2gfq0, q1, q3g 1 fq0, q1, q2gfq0, q1, q2g 0 fq0, q1, q2gfq0, q1, q2g 1 fq0, q1, q2gfq0, q1, q2, q3g 0 fq0, q1, q2, q3gfq0, q1, q2, q3g 1 fq0, q1, q2, q3g

Figure S.4.2 The next-state table of the DFSM equivalent to that given in Fig S.4.2.

state and the FSM advances from q1 to q2, from q2 to q3, and from q3 to q4, oninput 1 and moves from each of these states to q5 on 0. It returns to q5 on 0 and 1.By inspection we can see that the regular expression describing strings other than11 and 111 is (0 + 10 + 110 + 1110)(0+ 1)�.

b) The first position contains 1. If the string has odd length, it ends in 1 and isrepresented by 1((0 + 1)1)�. If it has even length, it is represented by 1((0 +1)1)�(0 + 1). Thus, all such strings are represented by 1((0 + 1)1)�(�+ 0 + 1).

4.10 Problem: Give regular expressions for the languages over the alphabet f0, 1, 2, 3, 4, 5,6, 7, 8, 9g describing positive integers that are:

a) even

b) odd

c) a multiple of 5

d) a multiple of 4

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Let � = f0, 1, 2, 3, 4, 5, 6, 7, 8, 9g.a) ��(2 + 4 + 6 + 8)

b) ��(1 + 3 + 5 + 7 + 9)

c) ��5

d) We assume that the digits are supplied least significant digit first. We note thatif the sequence has three or more digits, the integer can be written in the formq � 100 + r where q is the quotient and r the remainder under division by 100.Since 100 is divisible by 4, the integer is divisible by 4 if and only if the remainderis. Thus, we need only represent sequences of length 1 or 2 that are divisible by 4.

4.1 Solutions 63

The sequences of length 1 are (0+4+8). By enumeration we can see that those oflength 2 either have 0, 4 or 8 as the least significant digit and an even second leastsignificant digit or have 2 or 6 as the least significant digit and an odd second leastsignificant digit. Let E = f0, 2, 4, 6, 8g andO = f1, 3, 5, 7, 9g. It follows that theintegers divisible by 4 are denoted by (0+4+8)+(0+4+8)E��+(2+6)O��.

4.11 Problem: Give proofs for the rules stated in Theorem 4.3.1.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: In the following let R, S and T be the sets denoted by the regular expressionsr, s, and t.

(1) The concatenation of any set with the empty set ; is the empty set.

(2) The concatenation of any set R with the set containing the empty string � is theset itself.

(3) The union of any set R with the empty set is R itself.

(4) The union of any set R with R itself is R.

(5) The union of two sets R and S is commutative; that is, R [ S = S [R.

(6) R(S [ T ) = RS [ RT because the concatenation of a set R with the union,S[T , of two sets is the set of strings that have R as prefix and strings from eitherS or T as suffix.

(7) (R[S)T = RT [ST because the concatenation of the union of two sets, S[T ,with a set R is the set of strings that have R as suffix and strings from either S orT as prefix.

(8) R(ST ) = (RS)T because the concatenation operation is associative.

(9) ;� = � because the Kleene closure contains � and the concatenation of the empty

set with itself, which is the empty set.

(10) �� = � because the Kleene closure contains � and the concatenation of the emptystring with itself, which is the empty string.

(11) (�+ r)+ = r� because the expression on the left is equal to (�+ r)+ (�+ r)2 +� � �+ (� + r)k + � � � and � and r are in r� and, by induction on k, (� + r)k iscontained in r + r2 + � � �+ rk.

(12) (�+ r)� = r� follows from (11) since (�+ r)� and (�+ r)+ differ only in thatthe former is guaranteed to contain �.

(13) We establish r�(�+ r) = r�, the other identity being shown in the same fashion.We observe that r�� = r� and r� = r+. Applying (6) we have the desired result.

(14) r�s+ s = r�s follows because s is included in r�s.

(15) r(sr)� = (rs)�r follows because the former can be written as r(� + (sr) +(sr)2 + � � � ) and the latter can be written as (�+ (rs) + (rs)2 + � � � )r. Clearly,r� = �r and r(sr)k = (rs)kr.

(16) We show that (r + s)� = (r�s)�r�, the other identity being shown in a similarfashion. Note that (r+ s)2 = r2 + rs+ sr + s2. Because (r+ s)k is the unionof all sequences of length k over the set fr, sg, (r + s)� contains alternations ofsequences of r’s and s’s.


Now consider (r�s)�r�. We write r�s as follows:

r�s =Xk1

rk1s

Thus, (r�s)� can be written as follows:

(r�s)� = �+Xk1

rk1s +Xk1,k2

rk1srk2s +X

k1,k2,k3

rk1srk2srk3s + � � �

That is, (r�s)� is alternating sequence of r’s and s’s ending in s. Concatenatingthis with r� on the right, we have the desired conclusion.

4.12 Problem: Show that �+01+(010)(10+010)�(�+1+01) and (01+010)� describethe same language. (The statement of this problem in the book was in error.). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Let p = � + 01 + (010)(10 + 010)�(� + 1 + 01). Using the rule thatr(s + t) = rs+ rt we have the following:

p = �+ 01 + (010)(10+ 010)� + (010)(10+ 010)�(1 + 01)

Using the rule r(sr)� = (rs)�r we rewrite (10 + 010)�(1 + 01) in the last term onthe right as follows:

(10+ 010)�(1 + 01) = ((1 + 01)0)�(1 + 01)

= (1 + 01)(0(1+ 01))�

= (1 + 01)(01+ 001)�

Similarly, we rewrite 0(10 + 010)� in the third term as follows:

0(10 + 010)� = 0((1 + 01)0)�

= (0(1 + 01))�0

= (01 + 001)�0

We then rewrite p as shown below.

p = � + 01 + (01)(01 + 001)�0 + (010)(1 + 01)(01+ 001)�

= � + 01 + (01)(01 + 001)�0 + (01)(01 + 001)(01+ 001)�

= � + 01(�+ (01 + 001)�0 + (01 + 001)(01+ 001)�)

= � + 01((01 + 001)�0 + (01 + 001)�)

= � + 01(01+ 001)�(0 + �)

= � + 01((�+ 0)01)�(0 + �)

= � + (01(�+ 0))�(01)(0 + �)

= � + (01 + 010)�(010 + 01)

= (01 + 010)�

4.1 Solutions 65

q1

q2

q3

q4

q5

0

1

0

0

0

0 0

0

0

Figure S.4.3 NFSM associated with Problem 4.13(a).

REGULAR EXPRESSIONS AND FSMS

4.13 Problem:

a) Find a simple nondeterministic finite-state machine accepting the language (01 [001 [ 010)� over � = f0, 1g.

b) Convert the nondeterministic finite state machine of part (a) to a deterministicfinite-state machine by the method of Section 4.2.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) We begin by constructing a 5-state NFSM that recognizes 01, 001, and 010. Wethen transform it into a machine recognizing (01 [ 001[ 010)� over � = f0, 1gusing the procedure given in Theorem 4.4.1. The resulting NFSM is shown inFig. S.4.3.

b) Let fi, : : : , kg denote the set fqi, : : : , qkg. The following table shows the setsof states that arise when applying this method to the NFSM of Fig. S.4.3. Theresulting DFSM is shown in Fig. S.4.4.

State State Set State State Set

a f1g b f2, 3gc f2g d f4ge f2, 3, 5g f ;

4.14 Problem:

a) Let � = f0, 1, 2g, and let L be the language over � that contains each stringw ending with some symbol that does not occur anywhere else in w. For exam-ple, 011012, 20021, 11120, 0002, 10, and 1 are all strings in L. Construct anondeterministic finite-state machine that accepts L.

b) Convert the nondeterministic finite-state machine of part (a) to a deterministicfinite-state machine by the method of Section 4.2.


a

b

f

c

e0

0

01

0 1

1

1

0

0, 1

1

d

Figure S.4.4 NFSM associated with Problem 4.13(b).

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) After fixing the last letter in a string, all strings over the remaining symbols in thealphabet are possible. Thus, the following regular expression describes such strings:

(� � 1)�1 + (�� 2)�2 + (�� 3)�3

Here � � i is the set of letters in � other than i.An NFSM for this set of expressions is shown in Fig. S.4.5.

b) Applying the the method of Section 4.2 to the NFSM of Fig. S.4.5 we create statesassociated with sets of its states. Let fi, : : : , kg denote the set fqi, : : : , qkg. Thecorresponding DFSM shown in Fig. S.4.6.

4.15 Problem: Describe an algorithm to convert a regular expression to an NFSM using theproof of Theorem 4.4.1.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: A recursive procedure CONSTRUCT to convert a regular expression (r.e.)into an NFSM parses the r.e. If it contains a single non-terminal, it constructs a simpleNFSM as shown in Fig. 4.6. If the r.e. is compound, it identifies the outermost opera-tor, applies CONSTRUCT to the component(s) of the compound expression and thenassembles the complete machine from the machine(s) generated by CONSTRUCTusing the rules given in the proof of the theorem.A representation of NFSMs is needed to insure that the assembly of a larger NFSMfrom smaller ones is easy. This is facilitated by making explicit in the representation theinitial and final states.

4.1 Solutions 67

q2

q3

q4

q1 q5

2, 3

1, 3

1, 2

1

2

3

1, 2

1, 3

2, 3

1, 2, 3

Figure S.4.5 An NFSM for Problem 4.14(a).

1, 2, 3

f1g

f3, 4, 5g

f2, 4, 5g

f2, 3, 5g

f2, 3g

f2, 5g

f2, 4g

f3, 5g

f4, 5g

f3, 4g

1

2

3

1

2

3

1

2

3

3

2

1

3

2

2

1

1

2

f4g

f5g

f3g

f2g

1, 2

1, 2

2, 3

1, 3

2

3

1

2, 3

1, 3

1, 2, 3

2

3

1

3

1

3

Figure S.4.6 A NFSM for Problem 4.14(b).


4.16 Problem: Design DFSMs that recognize the following languages:

a) a�bca�

b) (a+ c)�(ab+ ca)b�

c) (a�b�(b+ c)�)�

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: For each of these problems we used the techniques of Theorem 4.4.1. Wedo not detail them because they are straightforward.

a) This DFSM M has four states, a reject state, qR, and three other states, an initialstate q1 to which M returns on input a, and state q2 reached from q1 on input b,q3, the final state, reached from q2 on input c. M returns to q3 on input a. Allother transitions from these states are to state qR.

b) This DFSM M has five states, a reject state, qR, and four other states, an initialstate q1 to which M returns on input a or c, and states q2 reached from q1 on inputb and q3 reached from q1 on input c. M enters q4, the final state, from q2 on inputb or from q3 on input a. M returns to q4 on input b. All other transitions fromthese states are to state qR.

c) We construct a DFSM M with five states that accepts a�b�(a + b)�. M has areject state, qR, and four other states q1, q2, q3, and q4, all of which are final states.q1 is the initial state. M enters q2 from q1 on input a and returns to q2 on inputa. It enters q3 from q2 on input b and returns to q3 on input b. M enters q4 oneither b or c from q3 and returns to q4 on input b or c. The machineM� accepting(a�b�(b+ c)�)� is constructed in the standard way.

4.17 Problem: Design an FSM that recognizes decimal strings (over the alphabet f0, 1, 2,3, 4, 5, 6, 7, 8, 9g representing the integers whose value is 0 modulo 3.

Hint: Use the fact that (10)k = 1 mod 3 (where 10 is “ten”) to show that (ak(10)k+ak�1(10)k�1 + � � �+ a1(10)1 + a0) mod 3 = (ak + ak�1 + � � �+ a1 + a0) mod 3.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: We design a DFSM M that uses the fact given in the hint. It has threestates corresponding to the running value of (ak + ak�1 + � � �+ a1 + a0) mod 3 as afunction of k. (See Fig. S.4.7.) The states are q0, q1, and q2. q0 is the initial state andthe only final state. Thus, as expected, the empty string is accepted. The values of eachinput, ai, are grouped into three classes, those that have value 0 modulo 3, namely,f0, 3, 6, 9g, those that have value 1 modulo 3, namely, f1, 4, 7g, and those that havevalue 2 modulo 3, namely, f2, 5, 8g. Let a0, a1, and a2 denote regular expresions forthe value in each of the corresponding sets. Then, M returns to qi on input a0, forj = (i+1) mod 3 advances from qi to qj on a1, and for j = (i+2) mod 3 advancesfrom qi to qj on a2.

4.18 Problem: Use the above FSM design to generate a regular expression describing thoseintegers whose value is 0 modulo 3.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: For this problem increase the index on each state by 1; that is, qj becomesqj+1 for 0 � j � 2. If we use the approach of Section 4.4.2, we develop a regularexpression describing the strings that take the FSM M from state qi to qj for each i and

4.1 Solutions 69

a1 = f1, 4, 7ga2 = f2, 5, 8g

a0 = f0, 3, 6, 9gq1q0 q2

a2

a0a1

a2

a0a1

a2

a0

a1

Figure S.4.7 The NSFM designed in Problem 4.17.

j. Since only state q1 is an accepting state, we will do unnecessary work if we computeall the entries in R

(k)i,j for each k. Since r(3)1,1 is defined as shown, we work backward

from the expression to find the regular expressions needed to solve this problem.

r(3)1,1 = r

(2)1,1 + r

(2)1,3

�r(2)3,3

��

r(2)3,1

This implies that we need only compute r(2)1,1 , r(2)1,3 , r(2)3,3 , and r(2)3,1 . The expressions forthese regular expressions are given below.

r(2)1,1 = r

(1)1,1 + r

(1)1,2

�r(1)2,2

��

r(1)2,1

r(2)1,3 = r

(1)1,3 + r

(1)1,2

�r(1)2,2

��

r(1)2,3

r(2)3,3 = r

(1)3,3 + r

(1)3,2

�r(1)2,2

��r(1)2,3

r(2)3,1 = r

(1)3,1 + r

(1)3,2

�r(1)2,2

��

r(1)2,1

The values of r(1)i,j are computed from r(0)i,j which are read from the state diagram for

the machine M . The values of r(0)i,j and r(1)i,j are shown below. We compute r(1)i,j using

r(1)i,j = r

(0)i,j + r

(0)i,1 (r

(0)1,1 )

�r(0)1,j .

r(0)i,j 1 2 3

1 �+ a0 a1 a2

2 a2 �+ a0 a1

3 a1 a2 �+ a0

r(1)i,j 1 2 3

1 a�0 a�0a1 a�0a2

2 a2a�

0 �+ a0 + a2a�

0a1 a1 + a2a�

0a2

3 a1a�

0 a2 + a1a�

0a1 �+ a0 + a1a�

0a2


We then have the following expressions for the regular expressions in question.

r(2)1,1 = a�0 + a�0a1(a0 + a2a

�

0a1)�a2a

�

0

r(2)1,3 = a�0a2 + a�0a1(a0 + a2a

�

0a1)�(a1 + a2a

�

0a2)

r(2)3,3 = (�+ a0 + a1a

�

0a2) + (a2 + a1a�

0a1)(a0 + a2a�

0a1)�(a1 + a2a

�

0a2)

r(2)3,1 = a1a

�

0 + (a2 + a1a�

0a1)(a0 + a2a�

0a1)�a2a

�

0

The expression for r(3)1,1 can be composed from these expressions.Another approach to this problem that can be simpler is to associate the regular expres-sion Ei with the input strings that carry M from the initial state to the state qi. Wecan then write the following equations for these regular expressions which show howone r.e. can be expresed in terms of the other r.e.’s in one-step.

E1 = �+ E1a0 +E2a2 + E3a1

E2 = E1a1 + E2a0 +E3a2

E3 = E1a2 + E2a1 +E3a0

The empty string appears on the righthand side of the first equation because state q1 isthe initial state which means that M can reach q1 from the initial state on the emptyletter. An equation of the form for the r.e. E

E = �+ E�

denotes strings in � plus the strings denoted by E concatenated with the strings in�. That is, E has as solution E = �(�)�. To see this, consider a string in �(�)�.It follows that such strings are either in � or in ��k for some k � 1. Thus, theysatisfy E = � + E�. On the other hand, consider a string w in the set E satisfyingE = �+E�. It follows that w is either in � or in E�. To be in E�, it must be in Econcatenated with �. But this means that it is either in �� or inE(�)2. Continuing inthis fashion, we see that w is in �(�)�. Thus, the solution to E = �+E� are exactlythe strings in �(�)�.This observation can be used to solve the above equations. That is,

E3 = (E1a2 +E2a1)a�

0

Substituting this into the equation for E2 we have

E2 = E1a1 + E2a0 + (E1a2 + E2a2)a�

0a2

= E1(a1 + a2a�

0a2) + E2(a0 + a2a�

0a2)

It follows that the solution to E2 is the following.

E2 = E1(a1 + a2a�

0a2)(a0 + a1a�

0a2)�

Substituting this expression for E2 into E3, it can be expressed as follows:

E3 = E1(a2a�

0 + (a1 + a2a�

0a2)(a0 + a1a�

0a2)�a1a

�

0)

4.1 Solutions 71

Substituting the expressions for E2 and E3 into the equation for E1 we have the fol-lowing expression for E1.

E1 = �+E1(a0 + (a1 + a2a�

0a2)(a0 + a1a�

0a2)�a2+

(a2a�

0 + (a1 + a2a�

0a2)(a0 + a1a�

0a2)�a1a

�

0)a1)

= �+E1(a0 + (a1 + a2a�

0a2)(a0 + a1a�

0a2)�(a2 + a1a

�

0a1) + a2a�

0a1)

= ((a0 + a2a�

0a1) + (a1 + a2a�

0a2)(a0 + a1a�

0a2)�(a2 + a1a

�

0a1))�

This expression for E1 is different from that derived earlier. Showing the equivalencebetween regular expressions can be challenging.

4.19 Problem: Describe an algorithm that constructs an NFSM from a regular expression r

and accepts a string w if w contains a string denoted by r that begins anywhere in w.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The solution to this problem is a DFSM that accepts all strings that containa string in the set denoted by r. Let M be a DFSM that accepts strings denoted by r.We construct an NFSM for the problem by adding to each state of M an edge backto M ’s initial state for each input letter. This machine nondeterministically decides oneach input letter whether to restart the search for a letter in r. Thus, if the input stringcontains a substring denoted by r, this NFSM will accept it.

THE PUMPING LEMMA

4.20 Problem: Show that the following languages are not regular:

a) L = fanban j n � 0gb) L = f0n12n0n j n � 1gc) L = fanbncn j n � 0g. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) Let L be recognized by an m-state FSM. Apply the pumping theorem to a stringwith n � m. Pumping up the string adds as to the beginning of the string,violating the condition that the number of as before and after b is the same.

b) Same as in (a).c) Same as in (b).

4.21 Problem: Strengthen the pumping lemma for regular languages by demonstrating thatif L is a regular language over the alphabet � recognized by a DFSM with m states andit contains a string w of length m or more, then any substring z of w (w = uzv) oflength m can be written as z = rst, where jsj � 1 such that for all integers n � 0,urs

ntv 2 L. Explain why this pumping lemma is stronger than the one stated in

Lemma 4.5.1.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The only change to the proof of Lemma 4.5.1 is to observe that the numberof states entered by the DFSM immediately before the first letter in z and followingeach letter of this string is m + 1. By the pigeonhole principle some state among thesestates is repeated. The substring s that takes the DFSM from this state to itself can


be removed or inserted into z an arbitrary number of times and the DFSM will reachexactly the same final state. Thus, the original string w is accepted if and only if thestring ursntv is accepted.

4.22 Problem: Show that the language L = faibj j i > jg is not regular.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Apply the version of the pumping lemma given in Problem 4.21 to thesubstring bj in aibj . If the language is regular, then by pumping up bj sufficientlymany times the new suffix will have more b’s than a’s, contradicting the definition ofthe language.

4.23 Problem: Show that the following language is not regular:

a) funzvmzwn+m j n,m � 1g. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) Apply the version of the pumping lemma given in Problem 4.21 to the substringun for n equal to the number of states of the DFSM that is supposed to recognizethis string. Clearly, by deleting the appropriate substring s the resulting string doesnot have a number of u’s and v equal in number to the number of w’s.

PROPERTIES OF REGULAR LANGUAGES

4.24 Problem: Use Lemma 4.5.1 and the closure property of regular languages under inter-section to show that the following languages are not regular:

a) fwwR j w 2 f0, 1g�gb) fww j where w denotes w in which 0’s and 1’s are interchangedgc) fw j w has equal number of 0’s and 1’sg. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) If fwwR j w 2 f0, 1g�g is regular, then its intersection with a regular languageis also regular. Intersect it with the regular language 0�1�0�. The result is thelanguage f0n12m0ng. By the pumping lemma, the substring 0n can be pumpedup to give a string not in the language. Contradiction.

b) We apply the same reasoning as in (a) except that we intersect fwwg with theregular expression 0�1�. The result is the language f0n1ng which is not regular.

c) As in (b), intersect fw j w has equal number of 0’s and 1’sg with 0�1� givingf0n1ng which is not regular.

4.25 Problem: Prove or disprove each of the following statements:

a) Every subset of a regular language is regular

b) Every regular language has a proper subset that is also a regular language

c) If L is regular, then so is fxy j x 2 L and y 62 Lgd) If L is a regular language, then so is fw : w 2 L and wR 2 Lge) fw j w = w

Rg is regular

4.1 Solutions 73

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) False. The language f0n1ng, which is not regular, is a subset of 0�1�.

b) True. f;g is a proper subset of all regular languages.c) False. The language f0n1m0pg is regular, as is easily shown. However, the lan-

guage formed by intersecting strings in this set with the reverse of strings in thisset is the language f0n1m0ng which can easily be shown not to be regular by thepumping lemma.

d) False. let 0 and 1 be two letters in the alphabet of L = fw j w = wRg. If L

is regular, intersect it with the regular language 0�1�0�. The result is the languagef0n12m0ng, which is not regular.

STATE MINIMIZATION

4.26 Problem: Find a minimal-state FSM equivalent to that shown in Fig. S.4.8.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: To find the minimal-state FSM we construct the �n for n � 0. Clearly,�0= ffq0, q1, q2g, fq3gg. Since the successors of q0 and q2 under both inputs are inthe same equivalence class of �0, but the successor to q1 under input 1 is in a differentequivalence class from the successors to q0 and q2, �1= ffq0, q2g, fq1g, fq3gg. Sincethe successors to q0 and q2 under input 0 are in different equivalence classes of �1, itfollows that �2= ffq0g, fq1g, fq2g, fq3gg. Since this equivalence relation cannot befurther refined, we find that the FSM in question is itself a minimal machine.

4.27 Problem: Show that the languages recognized by M and M� are the same, where �is the equivalence relation on M defined by states that are indistinguishable by inputstrings of any length.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

0

10

0

Start

q3q2

q0 q1

11

0

Figure S.4.8 A four-state finite-state machine.


Solution: Two states p and q of M are equivalent (they satisfy p � q) if and only iffor all input strings u 2 �� of all lengths either both Æ(p,u) and Æ(q,u) are in F (thefinal states) or both are not in F . Since the strings that are accepted by M (M�) arethose strings that takeM (M�) from its initial state to a state in F , it follows that thosestrings that are accepted by M are the same as those accepted by M�.

4.28 Problem: Show that the equivalence relation RL is right-invariant.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: u,v 2 �� are equivalent under RL if and only if for each z 2 ��, eitherboth uz and vz are in L or both are not in L.An equivalence relation R is right-invariant if for all w 2 ��, uwRvw.

To show thatRL is right-invariant we must only show that for allw 2 ��, uwRLvw.But for uwRLvw requires that either both uwz and vwz are inL or both are not inL. But this is guarenteed by the definition of RL since wz is just another string over��.

4.29 Problem: Show that the equivalence relation RM is right-invariant.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: uRMv if and only if Æ(s,u) = Æ(s,v). But RM is right-invariant if forall w 2 ��, uwRMvw. However, by the definition of RM the state reached by Munder uw and vw are the same which implies that RM is right-invariant.

4.30 Problem: Show that the right-invariance equivalence relation (defined in Definition 4.7.2)for the language L = fanbn j n � 0g has an unbounded number of equivalenceclasses.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: If the equivalence relation RL for L had a finite number of equivalenceclasses, L would be recognized by an FSM, which is impossible since L is not regularby the pumping lemma.

4.31 Problem: Show that the DFSM in Fig. 4.20 is the machine ML associated with thelanguage L = (10�1 + 0)�.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: We construct a machine for L and then find a minimal-state machine equiv-alent to it. We first construct a machine for 10�0 + 0 and then build one for L. AnNFSM machine for 10�0+0 has states q0, q1, and q2. q0 is the initial state which has atransition to q1 on 1 and to q2 on 0. The transitions from q1 are to q2 on 1 and to itselfon 0. q2 is an accepting state. The machine constructed to realize L has a new initialstate q3 which is accepting and has transitions to q1 on 1 and to q2 on 0. The transitionsfrom q2 are to itself on 0 and to q1 on 1. The equivalence relation R� on this machineis easily seen to have two equivalence classes, ffq2, q3g, fq1gg, with transitions fromthese classes that are isomorphic to those given in Fig. 4.20. The initial state is fq2, q3gand the final state is fq1g.

PUSHDOWN AUTOMATA

4.32 Problem: Construct a pushdown automaton that accepts the following language: L =fw j w is a string over the alphabet � = f(, )g of balanced parenthesesg.

4.1 Solutions 75

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: A sketch of this PDA: whenever a left parenthesis, (, is seen, push it onto thestack. Whenever a right parenthesis, ), is seen, pop the stack. If ever the blank inputsymbol is encountered when the stack is non-empty or the empty stack is encounteredwhen the tape symbol is non-blank, enter the reject state. Otherwise, enter the acceptstate.

4.33 Problem: Construct a pushdown automaton that accepts the following language: L =fw j w contains more 1’s than 0’sg.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The PDA keeps track of the difference in the number of 0’s and 1’s. Pushthe first letter on the stack. If it is 1, continue to push 1’s, popping a 1 when a 0 isencountered. If the first letter is 0, continue to push 0’s, popping when a 1 is encoun-tered. Initially and at all subsequent times the number of characters on the stack is thedifference in the number of 0’s and 1’s. It follows that if the blank input symbol isencountered on the input tape when the stack either empty or has a 1 at the top, thenthe input has more 1’s than 0’s. Otherwise, it has more 0’s than 1’s.

PHRASE STRUCTURE LANGUAGES

4.34 Problem: Give phrase-structure grammars for the following languages:

a) fww j w 2 fa, bg�gb) f02i j i � 1g. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) Let G = (N ,T ,R, S), where N = fS, A, B, C, D, L, R, Tg, T = fa, bg and Rconsists of the following rules:

a) S ! LR

b) L ! aLA

c) L ! bLB

d) L ! T

e) TA ! TC

f) TB ! TD

g) CA ! AC

h) DA ! AD

i) CB ! BC

j) DB ! BD

k) CR ! Ra

l) DR ! Rb

m) TR ! �

The first rule places R on the right. It then rewrites L as a string w of terminals fol-lowed by the string W which mirrorsw in that the terminals a and b are replacedby the non-terminals A and B. Rule d) starts a process that reverses the order of thenon-terminals in W . Rules e) and f ) convert A and B into C and D, respectively,which by rules g)-j) pass to the right of each letter inW until they reach R. At thispoint they are converted by rules k) and l) to a and b respectively and R is shiftedleft. Since no rule is given to rewrite strings of C and D, if a string of terminals isto be produced, all instances of these non-terminals must migrate to the right endof the string.

b) Let G = (N ,T ,R, S), where N = fS, A, F, Tg, T = f0g and R consists of thefollowing rules:


a) S ! FT

b) F ! FA0

c) F ! �

d) A0 ! 00A

e) AT ! T

f) T ! �

The rule F ! FA0 generates instances of A each of which doubles the number ofinstances of 0 to its right using the rule A0! 00A. This insures that the numberof 0’s is equal to 2k where k is the number of instances of A that are generated.

4.35 Problem: Show that the following language can be described by a phrase-structuregrammar:

fai j i is not primeg

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: Because i, the number of instances of a, is not a prime, we give a grammarthat first generates a representation in unary notation for two or more integers. Wethen give rules that form the product of the rightmost two integers, leaving the resultin unary notation. These rules also from the product of the remaining integers, if any,with the product produced so far.The grammar is G = (N ,T ,R, S), where N = fS, A, D, I, J, L, Q, Rg, T = fag andR consists of the following rules:

a) S ! LQJ

b) Q ! QQ

c) Q ! ID

d) I ! IA

e) I ! AA

f) J ! Ja

g) J ! aa

h) Ada ! dAa

i) Aa ! aaA

j) DD ! D

k) LD ! �

Rule a) must be followed by b) which together produce a string of the form LQm�1J.J and each instance of Q will be rewritten as integers � 2 in unary notation. Rule c)rewrites Q as I followed by the separator D. I is rewritten by rules d) and e) as Akr forsome integer kr � 2. Also, J is rewritten by rules f ) and g) as ak1 for some integerk1 � 2. Rules h) moves the non-terminal A past D when the terminal a is to its right.By rule i) each non-terminal copies each instance of a to its right. As a consequence,terminals are produced from right to left and the number of terminals resulting from m

consecutive integers is their product. After all non-terminals between two consecutiveinstances of D or to the left of the leftmost D move to the right of D, two consecutiveinstances of D result, which by rule j) is rewritten as D. Finally, rule k) annihilates theremaining instance of D. If this is not the last rule to be applied, the resulting stringwill not be a string of terminals.

CONTEXT-SENSITIVE LANGUAGES

4.36 Problem: Show that every context-sensitive language can be accepted by a linearbounded automaton (LBA), a nondeterministic Turing machine in which the tapehead visits a number of cells that is a constant multiple of the number of characters inthe input string w.

4.1 Solutions 77

Hint: Consider a construction similar to that used in the proof of Theorem 5.4.2.Instead of using a second tape, use a second track on the tape of the TM.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Consider a stringwwhose membership in a context-sensitive languageL(G)with grammar G is to be tested. Let it be written on the tape of an NTM. If w is inL(G), each rule of G that is used to derive w leads to strings whose lengths are non-decreasing. Thus, the length of each intermediate string is no more than the length ofw. As a consequence, we can use a second track on the tape of the LBA to write out thepossible intermediate strings leading to w. Initially the start symbol is placed beneaththe leftmost input symbol and rules are applied nondeterministically. Substrings mayneed to be shifted right to provide space for a rewriting, a task readily performed in thespace available. If a string is produced that is longer than w, contains non-terminalsthat cannot be further expanded, or is a terminal string that is not equal to w, itis rejected. Otherwise, it is accepted. It follows that if w 2 L(G), there exists anondeterministic LBA that accepts it. Since each track of a tape is equivalent to a fixed-length expansion of the original tape, a context-free language can be accepted by anLBA.

4.37 Problem: Show that every language accepted by a linear bounded automaton can begenerated by a context-sensitive language.

Hint: Consider a construction similar to that used in the proof of Theorem 5.4.1 butinstead of deleting characters at the end of TM configuration, encode the end markers[ and ] by enlarging the tape alphabet of the LBA to permit the first and last charactersto be either marked or unmarked.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Two ideas are used in this proof beyond those used in Theorem 5.4.1,namely, the number of tape cells used is a constant multiple of the length of the in-put, and the end markers [ and ] can be encoded by extending the tape alphabet. Allof the rules of a phrase structure grammar except for b)-e) are length preserving or in-creasing. Rules b) and d) are not necessary because the assumption that the TM is alinear bounded automaton. Rules c) and e) can be eliminated by expanding the tapealphabet.

4.38 Problem: Show that the grammarG1 in Example 4.9.1 is context-sensitive and gener-ates the language L(G1) = fanbncn j n � 1g.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Grammar G1 is context-sensitive because each rule is length preserving orincreasing. To show that it generates fanbncn j n � 1g, observe that rule a) is the firstto be applied. Before S is is replaced by b) for the last time, S can be replaced multipletimes by a). While other rules can be applied after one application each of a) and b),there is no loss in generality in assuming that the first rules applied consist of k � 1applications of a) followed by one of b) yielding the derivation C

�) ak(BC)k. Rulec) or d) can be applied to this result. If rule f ) is applied and the character on the rightof C is not C, the rewriting halts. Thus, the derivation produces a terminal string onlyif all instances of C move to the right of all instances of B. Thus, terminal strings are


derived only if rule c) is applied k times to move instances of C to the right of instancesof B. This action has the effect of generating the string of the form anbncn.

4.39 Problem: Show that the language f02i j i � 1g is context-sensitive.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Let G = (N ,T ,R, S), where N = fS, A, F, Tg, T = f0g and R consistsof the following rules:

a) S ! 0

b) S ! FT

c) F ! A

d) F ! FA

e) AT ! 00

f) A0 ! 00A

Rule a) generates 0. Rules b), c) and d) are used to generate longer strings. Rules b)and c) can only be used once. If rule d) is used k � 1 times, the string derived from S

will contain k instances of A. Each instance of A doubles the number of 0’s to its rightwhere T serves not only as a terminator but a representative for one instance for 0. Itfollows that strings of 2k 0’s are generated by this grammar.

4.40 Problem: Show that the context-sensitive languages are closed under union, intersec-tion, and concatenation.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Without loss of generality let G1 and G2 be context-sensitive grammarswith disjoint sets of non-terminals. Each of the rules in their grammars is not lengthdecreasing. Let S1 and S2 be the start symbols of these grammars.

A grammar generating the union of L(G1) and L(G2) has a new non-terminal S3 andthe rules S3 ! S1 and S3 ! S2 as well as those in G1 and G2. The new rules are notlength decreasing and they generate the terminal strings in one or the other or both ofthe languages.

To show that CSL’s are closed under intersection, create two tracks on the tape of aLBA and write the input string,w, on both tracks. On the first track simulate the LBArecognizing L(G1). If it accepts w, repeat the process on the second track with theLBA accepting L(G2), acceptingw if this LBA accepts the string. The strings acceptedby both machines are in the intersection of L(G1) and L(G2). Since the two tracksexpand by a constant factor the amount of storage needed to recognize either language,the intersection is also recognized by an LBA.

For CFL’s we generated a grammar for their concatenation by adding a new non-terminal S3 and the rule S3 ! S1S2. Since rules are invoked only to replace singlenon-terminals, the fact that non-terminals are disjoint insures that replacements in thederivations from S2 and S3 are disjoint. However, in the case of CSL’s, since the set ofterminals may not be disjoint and CSL’s rewrite strings rather just single non-terminals,some rule may be invoked on the derivations from S1S2 that would not be invokedfrom just S1 or S2 alone. To fix this problem, in every derivation G1 that has a terminala replace it by a unique non-terminal Na and add the rule Na ! a. If a is also aterminal of G2, replace a by Ma where Ma is a non-terminal distinct from Na. Sincethe non-terminals of the two grammars are disjoint, it follows that the left-hand sideof every rule in both languages are disjoint. Thus, rules of G1 apply only to stringsderived from S1 and rules of G2 apply only to strings derived from S2.

4.1 Solutions 79

CONTEXT-FREE LANGUAGES

4.41 Problem: Show that language generated by the context-free grammar G3 of Exam-ple 4.9.3 is L(G3) = fcancancbmcbmc j n,m � 0g.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: After application of rule a), rule b) is applied some number of times, sayn � 1, (including 0) after which rule c) is applied. Interlaced with the application ofrules b) and c) can be rules d) and e) in which d) can be applied some number of times,say m�1, (including 0) after which rule e) is applied. Under these conditions balancedpairs of n instances of a are produced separated by c as are balanced pairs ofm instancesof b are produced separated by c. The latter follow the former.

4.42 Problem: Construct context-free grammars for each of the following languages:

a) fwwR j w 2 fa, bg�gb) fw j w 2 fa, bg�, w = w

Rgc) L = fw j w has twice as many 0’s as 1’sg. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) The strings in this language have even length

a) S ! aSa

b) S ! bSb

c) S ! �

b) The strings in this language can have even or odd length. Those of even length arewritten as xxR whereas those of odd length are written as xcxR for c equal to aor b. The following rules generate strings of these two types.

a) S ! aSa

b) S ! bSb

c) S ! a

d) S ! b

e) S ! �

c) The rule S ! 001S generates twice as many 0’s as 1’s but the former precede thelatter. To generate strings that have twice as many 0’s as 1’s, a grammar suffices thathas one rule such as the one above for each permutation of the four characters onthe right-hand side plus the rule S! �.

4.43 Problem: Give a context-free grammars for each of the following languages:

a) fw 2 fa, bg� j w has twice as many a’s as b’sgb) farbs j r � s � 2rg. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) Same as given in the solution to Problem 4.42(c) except that 0 is replaced by a and1 by b.


b) a) S ! aSb

b) S ! aSbb

e) S ! �

REGULAR LANGUAGES

4.44 Problem: Show that the regular language G4 described in Example 4.9.4 is L(G4) =(01)�0.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: There is only one rule that can be applied after a), namely c). The compo-sition of these two rules generates the string 01B. Similarly, there is only one rule thatcan follow d), namely, c), implying that B can be rewritten either as 0 or 01B. Thus, thefollowing rules generate the same strings as do the rules of G4: a) S ! 0, b) S ! 01B,c) B ! 0, and d) B ! 01B. It follows that the language L(G4) contains 0 and stringsresulting from one application of b) and k � 1 instances of d). That is, the languagecontains only strings of the form (01)k0 for k � 0.

4.45 Problem: Show that grammar G = (N ,T ,R, S), where N = fA, B, Sg, T = fa, bgand the rules R are given below, is regular.

a) S ! abA

b) S ! baB

c) S ! B

d) S ! �

e) A ! bS

f) B ! aS

g) A ! b

Give a derivation for the string abbbaa.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The following rules generate this string: a), e), b), f ), and d).

To show that this grammar is regular, we demonstrate that it can be put into the stan-dard normal form, namely, right-hand sides that have a terminal followed by a non-terminal or a single terminal. We do this by replacing the rule S ! abA by the tworules S ! aD and D ! bA and replacing the rule S ! baB by the two rules S ! bEand E ! aB. Also, we delete the rule S ! � and add the rules obtained by deleting S

from the right-hand sides of rules containing it, namely, A! b and B! a.

4.46 Problem: Provide a regular grammar generating strings over f0, 1g not containing 00.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The rules S ! 0, S ! 1, S ! 0A, A ! 1S, and S ! 1S generatesthese strings because, unless the string 0 is generated, each time that 0 is generated it isfollowed by 1. Otherwise, no restrictions are placed on the strings.

4.47 Problem: Give a regular grammar for each of the following languages and show thatthere is a FSM that accepts it. In all cases � = f0, 1g.a) L = fw j the length of w is oddgb) L = fw j w contains at least three 1sg

4.1 Solutions 81

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) S ! 0 and S ! 1 generate strings of length 1. Using as our inductive hypothesisthat S generates strings of odd length, the following rules generate an arbitrarystring of even length followed by S, which satisfies our inductive hypothesis.

a) S ! 0

b) S ! 1

c) S ! 0A

d) S ! 1A

e) A ! 0S

f) A ! 1S

b) The rules A ! 0A, A ! 1A, and A ! � generate all strings over f0, 1g, as can beeasily shown. It follows that the rule S ! A1A1A1A generates all strings containingat least threes 1’s.

REGULAR LANGUAGE RECOGNITION

4.48 Problem: Construct a finite-state machine that recognizes the language generated bythe grammar G = (N ,T ,R, S), where N = fS, X, Yg, T = fx, yg, and R containsthe following rules: S! xX, S! yY, X! yY, Y! xX, X! �, and Y! �.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: An NFSM recognizing this language has states labled S, X, and Y. Transitionsfrom S to X and Y are made on inputs x and y, respectively. From X there is a transitionto Y on input y. From Y there is a transition to X on input x. To convert this NFSMto a DFSM, add an edge to a reject state from each of the states of this NFSM fortransitions that are missing. Both states X and Y are final states.

4.49 Problem: Describe finite-state machines that recognize the following languages:

a) fw 2 fa, bg� j w has an odd number of a’sgb) fw 2 fa, bg� j w has ab and ba as substringsg. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) The FSM of Fig. 3.2 with 0 and 1 replaced by b and a recognizes this language.

b) A string containing both ab and ba satisfies one of the following four conditions:a) ab appears before ba, b) ba appears before ab, c) it contains aba, or d) it con-tains bab. We design NFSMs for each condition and take the union of these fourNSFMs.

An NFSM for a) has five states, q1, q2, q3, q4, and q5. It can remain in q1 on inputsa and b or can enter q2 on a. It then moves to q3 on input b, possibly remainingthere on either input and then moving to q4 on b followed by a move to q5 on a

where it remains on all inputs. An NFSM for b) has the same structure except thatthe role of a and b are interchanged. An NFSM for c) has four states. It can remainin the first one one each input or move to the second, third and fourth states oninputs a, b and a, respectively, where it remains on all inputs. An NFSM for d) hasthe same structure as that for c) except that the role of a and b are interchanged.An NSFM for all four cases is obtained by coalescing the initial and final states foreach of the machines.


4.50 Problem: Show that, if L is a regular language, then the language obtained by reversingthe letters in each string in L is also regular.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: If the DFSM M recognizing L has a single final state, a DFSM MR for thereverse of L is obtained by reversing the directions of all edges in M and letting M ’sfinal state be the initial state of MR. This machine, which is an NFSM, can enter MR’sfinal state on the input w if and only if the reverse of w will take M from its initialstate to M ’s final state. When M has more than one final state, reverse all the edges ofM to form MR. Let the initial state of MR be a new state that has an edge to everystate from which there is an edge to a final state of M . This initial state of MR allowsany final state of M to be reached by the original input. It follows that the reverse ofstrings in L is also regular.

4.51 Problem: Show that, if L is a regular language, then the language consisting of stringsin L whose reversals are also in L is regular.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The strings in question are both in L and its reverse. Because a regularlanguage and its reverse are both regular and the set of regular languages is closed underintersection, the intersection of a language with its reverse is regular.

PARSING CONTEXT-FREE LANGUAGES

4.52 Problem: Use the algorithm of Theorem 4.11.2 to construct a parse tree for the string(a � b + a) � (a + b) generated by the grammar G5 of Example 4.11.2, and give aleftmost and a rightmost derivation for the string.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: We exhibit fbi,i+j j 1 � i � 13, 1 � j � 7g below. Associated witheach non-terminal in square brackets are the right-hand sides of rules with these non-terminals on the left-hand side.i bi,i+1 bi,i+2 bi,i+3 b

i,i+4 bi,i+5 bi,i+6 bi,i+7

1 ( ; ; ; ; ; E,T,F[(C]

2 E, T, F ; E[TB], T[TD] ; E[EA] C,G,H[E)] ;

3 � B,D[�F] ; ; ; ; ;

4 E, T, F ; E[EA] C,G,H[E)] C,G,H[E)] ; ;

5 + A[+T] ; ; ; ; ;

6 E, T, F ; ; ; ; ; ;

7 ) C,G,H[E) ; ; ; ; ;

8 � ; ; ; ; B,D[�F]

9 ( ; ; ; E,F[(C]

10 E, T, F ; E[EA] C, G, H[E)]

11 + A[+T] ;

12 E, T, F C, G, H[E)]

13 )

All the entries bi,j for 8 � j � 12 are ;. The value of b1,14 is fE[TB], T[TD]g. Sincethis set contains the start symbol E, the string (a � b + a) � (a+ b) is in the language.We can read the derivation by following the path backward from the non-terminal E.

4.1 Solutions 83

T B

E

E

E

T

A

a

E

Ta

+

b

)

�

+B

+ a

+

E A

�G F

H( �

)

T

)

)

(

(

(

�

b

F

Figure S.4.9 The parse tree for Problem 4.52.

The parse tree for this string, see Fig. S.4.9.

4.53 Problem: Let G = (N ,T ,R, S) be the context-free grammar with N = S andT = f(, ), 0g with rules R = fS ! 0, S ! SS, S ! (S)g. Use the algorithm ofTheorem 4.11.2 to generate a parse tree for the string (0)((0)).

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: First, convert the grammar to Chomsky normal form.

S ! 0

S ! SS

S ! CB

C ! AS

A ! (

B ! )

Once the grammar is in Chomsky normal form construct the 9 � 9 matrices B(1),B(2), B(3), B(4), B(5), B(6), B(7) and B(8).

B(1) =

2666666666666666664

; fAg ; ; ; ; ; ; ;

; ; fSg ; ; ; ; ; ;

; ; ; fBg ; ; ; ; ;

; ; ; ; fAg ; ; ; ;

; ; ; ; ; fAg ; ; ;

; ; ; ; ; ; fSg ; ;

; ; ; ; ; ; ; fBg ;

; ; ; ; ; ; ; ; fBg

; ; ; ; ; ; ; ; ;

3777777777777777775


B(2) =

2666666666666666664

; ; fCg ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; fCg ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

3777777777777777775

B(3) =

2666666666666666664

; ; ; fSg ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; fSg ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

3777777777777777775

B(4) =

2666666666666666664

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; fCg ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

3777777777777777775

B(5) =

2666666666666666664

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; fSg

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

3777777777777777775

4.1 Solutions 85

B(6) =

2666666666666666664

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

3777777777777777775

B(7) =

2666666666666666664

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

3777777777777777775

B(8) =

2666666666666666664

; ; ; ; ; ; ; ; fSg

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

; ; ; ; ; ; ; ; ;

3777777777777777775

S 2 B(8)1,9 so the string (0)((0)) is in the language. Finally, draw the parse tree.

S

S S

C B

A S

( 0

)

C B

A S

C(

A S

( 0

B

)

)

CFL ACCEPTANCE WITH PUSHDOWN AUTOMATA

4.54 Problem: Construct PDAs that accept each of the following languages:

a) fanbn j n � 0gb) fwwR j w 2 fa, bg�gc) fw j w 2 fa, bg�, w = w

Rg


. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) If in state q0 the blank symbol is read, the PDA accepts the input. If in state q0 thesymbol b is read, the PDA rejects the input by entering qR from it never exits. Ifin state q0 a is read, it is pushed onto the stack and state q1 entered. If in state q1

a is read, a is pushed on the stack and state q1 entered. If in this state b is read andthe symbol on the stack is a, state q2 is entered and a is popped. If however, thestack is empty, state qR is entered.

b) This is an example of a language for which the potential nondeterminism of thePDA is needed.This PDA accepts the empty tape. If the tape is not empty, it enters q1, the “pushstate,” and pushes the input symbol onto the tape, returning either to the pushstate or entering the “pop state” nondeterministically. It remains in the pop statewhile the symbol on the top of the stack and the tape match. If both the tapeand the stack contain blank symbols, the accept state is entered. Otherwise, if thesymbol on the tape and the stack do not match, the string is rejected.

c) If a string in this language has even length, it is in the above language. Thus, itdiffers only when strings have odd length in which case there is a middle characteraround which the string is symmetric. Thus, we need only augment the abovemachine to decide nondeterministically whether the string has even or odd lengthand, if odd, when it has reached the middle of the string at which point it does notpush or pop on this character, popping after reading the next character.

4.55 Problem: Construct PDAs that accept each of the following languages:

a) fw 2 fa, bg� j w has twice as many a’s as b’sgb) farbs j r � s � 2rg. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) A PDA for this problem follows from the CFL grammar given in the solution toProblem 4.42(c) and the construction of Theorem 4.12.1.

b) A PDA that accepts strings of the form arbs for s � 2r pushes two a’s onto thestack for every a seen at the beginning of the string. At each b one character ispopped from the stack. If the blank is not popped, the string is of the desiredform. The strings arbs for r � s can be found by pushing only one a onto thestack for each a in the input. It would appear that the desired strings are in theintersection of these two languages. However, since the set of CFL’s is not closedunder intersection, it would appear that the language in question is not CFL, whichwe know from Problem 4.43(b) is not the case.The solution is to design a nondeterministic PDA that pushes either one or twoinstances of a for every a in the input and then pops one letter from the stack foreach b that is seen. If we accept only strings for which the number of a’s placed onthe stack is equal to the number of b’s, strings of the proper form can be acceptednondeterministically. Thus, the language is CFL.

4.56 Problem: Use the algorithm of Theorem 4.12.2 to construct a context-free grammarthat accepts the language accepted by the PDA in Example 4.8.2.

4.1 Solutions 87

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Space does not permit the enumeration of all the productions in the grammarin question. It contains the following rules:

1) S ! < s, �, f >

2) < s, �, s > ! �

3) < p, �, p > ! �

4) < f , �, f > ! �

5) < r, �, r > ! �

It also contains rules for the form < p, y, � >! x < q, z, � > for all � 2 Q and forall (p,x, y; q, z) 2 �, where y 6= �. These rules are shown below where

6) < s, a, � > ! b < p, �, � >

7) < p, a, � > ! b < p, �, � >

8) < p, , � > ! b < r, �, � >

9) < p, a, � > ! � < f , �, � >

10) < p, , � > ! � < f , �, � >

11) < s, , � > ! b < r, �, � >The rules of the form < p,u, � >! x < q, z, � >< � ,u, � > for all �, � 2 Q, for all(p,x, �; q, z) 2 �, and for all u 2 � [ f�g are shown below

12) < s,u, � > ! a < s, a, � >< � ,u, � >

13) < p,u, � > ! a < r, �, � >< � ,u, � >

14) < s,u, � > ! � < f , �, � >< � ,u, � >

15) < f ,u, � > ! � < f , �, � >< � ,u, � >

16) < r,u, � > ! � < r, �, � >< � ,u, � >

4.57 Problem: Construct a context-free grammar for the language fwcwR j w 2 fa, bg�g.Hint: Use the algorithm of Theorem 4.12.2 to construct a context-free grammar thataccepts the language accepted by the PDA in Example 4.8.1.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: The hint is too complex. Instead we give a direct construction.

S ! aSa

S ! bSb

S ! c

PROPERTIES OF CONTEXT-FREE LANGUAGES

4.58 Problem: Show that the intersection of a context-free language and a regular languageis context-free.Hint: Form machines accepting the two language types, construct a machine acceptingtheir intersection.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Consider a PDA, M , that accepts language L1 and a DFSM, N , that acceptsL2. Construct a new PDA, M �N that simulates the two machines in parallel. Con-


struct the new PDA so that ifM does not read its input, it does not advance the state ofN . Its control unit has states represented as pairs (m,n) where m is a state of M and nis a state of n. If M has a transition (m,x, y;m0, z) and N has a transition (n,x;n0),M � N is the PDA that has the transition ((m,n),x, y; (m0 ,n0), z) if x 6= � and((m,n),x, y; (m0 ,n), z) otherwise.An accept state of this new PDA is a pair (m,n) such that m is an accept state of Mand n is an accept state of N . This machine accepts its input if it is accepted by bothmachines. Thus, it accepts strings that are in the intersection of the languages acceptedby the two machines.

4.59 Problem: Suppose that L is a context-free language and R is a regular one. Is L � R

necessarily context-free? What about R� L? Justify your answers.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The language L�R contains the strings in L that are not in R. The stringsnot in R are the strings in the complement of R which is regular. Thus, L � R isthe intersection of L with the complement of R. By Problem 4.58 this intersection iscontext-free.

In general R � L is not context-free because if R = ��, where � is the alphabet overwhich L is defined, then R�L is the complement of L which may not be context-free.

4.60 Problem: Show that, if L is context-free, then so is LR = fwR j w 2 Lg:. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Construct a context-free grammar for LR using the grammar for L as a base.(We know L has a context-free grammar since L is context-free.) The grammar for LR

has the same non-terminals and terminals as L. Its rules are the same as L’s except thatthe right-hand side is reversed. For example, the rule S ! AB becomes S ! BA.And, of course, rules that go to a single element are unchanged.This grammar will generate strings that are the reverse of strings in L. We have success-fully constructed a CFG for LR so it is a CFL.

4.61 Problem: Let G = (N ,T ,R, S) be context-free. A non-terminal A is self-embeddingif and only if A

�)G sAu for some s,u 2 T .

a) Give a procedure to determine whether A 2 N is self-embedding.

b) Show that, if G does not have a self-embedding non-terminal, then it is regular.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution:

a) Let m be the number of non-terminals. For each non-terminal A we need onlyconsider sequences of at most m � 1 transitions because we are guaranteed that A

is repeated in this many transitions if it is repeated at all. Let r be the maximumnumber of non-terminals on the right-hand side of a rule. It follows that we needonly consider at most 1 + R + � � �+ rm�1 = (rm � 1)=(r � 1) derivations ofdepth m � 1 or less before discovering if A is self-embedding. We do this for eachnon-terminal to determine if the language itself is self-embedding.

b) Assume that G does not have a self-embedding non-terminal. This implies thatevery derivation for every non-terminal A either yields a terminal string or a stringof the form A

�)G sA (it is right-recursive) or A�)G Au (it is left-recursive)

4.1 Solutions 89

for terminal sequences u and v. Because a non-terminal that yields strings of bothtypes is self-embedding, this does not happen.Without loss of generality, let A be left-recursive. Let r denote the regular expres-sion that describes the set of terminal strings generated from A without embeddingand let s be the regular expression denoting u. Then, A

�)G rs� which impliesthat the strings derivable from A can be described by regular expressions.

CFL PUMPING LEMMA

4.62 Problem: Show that the following languages are not context-free:

a) f02i j i � 1gb) fbn2 j n � 1gc) f0n j n is a primeg. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) Assume the language is recognized by a CFL with m non-terminals. Let i be suchthat 2i � 2m�1 + 1. It follows that 02i = rstuv, that is, for positive integers aand b, s = 0a and u = 0b. From the pumping lemma it follows 02i+(n�1)(a+b).This is in the language if 2i + (n � 1)(a + b) is a power of two. For this to hold,we require that for each n � 0, (n � 1)(a+ b) = 2k � 2i for some k.We consider two cases, 1 + (a + b) = 2j for some j and 1 + (a + b) 6= 2j forany j. In the first case, let (n � 1) = 2i in which case 2i + (n � 1)(a + b) =2i(1+ 2(a+ b)) = 2i(2j+1� 1) which is not a power of two. In the second case,set (n� 1) = 2i in which case 2i + (n � 1)(a+ b) = 2i(1 + (a + b)) which isalso not a power of two.

b) For n sufficiently large, bn2= rstuv can be pumped up to rsptupv for p � 0.

Let s and u be of lengths a and b, respectively, where a + b > 0. If the languageis context-free, it follows that for each p � 0, n2 + (p � 1)(a+ b) = q2 for someinteger q. But if we choose p� 1 = n, the left-hand side is n(n+ a+ b) which isnot a square because n+ a+ b > n.

c) For n sufficiently large, 0n = rstuv can be pumped up to rsptupv for p � 0.Let s and u be of lengths a and b, respectively, where a + b > 0. If the languageis context-free, it follows that for each p � 0, n + (p� 1)(a+ b) is a prime. Butif p� 1 = n, n+ (p� 1)(a+ b) is not prime.

4.63 Problem: Show that the following languages are not context-free:

a) f0n1n0n1n j n � 0gb) faibjck j 0 � i � j � kgc) fww j w 2 f0, 1g�g. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) The proof is identical with that given for Lemma 4.13.2.b) Assume that faibjck j 0 � i � j � kg is context-free. Then by the Pumping

Lemma every sufficiently long string rstuv can be pumped up to rsptupv for


p � 0. Consider a sufficiently long string in which i = j = k. If s or u fallscompletely into the a’s or b’s, let p � 1 = i + 1 which insures that the new stringis not in the language. If either string falls completely into the c’s, pump u done,generating a string that violates the condition for membership in the language. Ifeither s or u overlaps a’s and b’s or overlaps b’s and c’s, pumping up the stringtwice (p = 2), results in b’s preceding a’s or c’s preceding b’s.

c) If L = fww j w 2 f0, 1g�g is context-free, its intersection with the regularlanguage 0�1�0�1� is the context-free language f0i1j0i1j j 0 � i, jg. Consider astring in this language of the form 0n1n0n1n = rstuv for n = 2m where m isthe number of non-terminals in the language. Then, jstuj � 2m. It follows thatstu overlaps with at most two of the four consecutive substrings 0n, 1n, 0n or 1n.If stu overlaps with one of them, rs2

tu2v is not of the form 0i1j0i1j for any i

and j. If it overlaps with two of the four substrings, rsptupv contain too manyalternations of 0’s and 1’s. Contradiction.

4.64 Problem: Show that the language fww j w 2 fa, bg�g is not context-free.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: This problem is identical with Problem 4.63(c) except for a change of alpha-bet.

CFL CLOSURE PROPERTIES

4.65 Problem: Let M1 and M2 be pushdown automata accepting the languages L(M1)and L(M2). Describe PDAs accepting their union L(M1) [ L(M2), concatenationL(M1) � L(M2), and Kleene closure L(M1)

�, thereby giving an alternate proof ofTheorem 4.13.1.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: For the union construct a control unit which from its initial state nondeter-ministically enters a state reached from the initial state of M1 or M2. Such a machinecan accept a string accepted by either PDA. It follows that it does accept strings in theirunion.For the concatenation, let the new PDA start in the start state of M1 and executetransitions in M1 until arriving at an accept state from which it makes a transition tothe start state of M2. This machine accepts strings in the concatenation of the twolanguages.

For the Kleene closure, the new PDA implements M1 except that edges that are tran-sitions from its final state are added to the states reached from its initial state, just aswas done with finite-state machines. This language recognizes the Kleene closure ofL(M1).

4.66 Problem: Use closure under concatenation of context-free languages to show that thelanguage fwwR

vRv j w,v 2 fa, bg�g is context-free.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: The language fwwR j w 2 fa, bg�g is context-free, as shown in Prob-lem 4.42(a). It follows that fvRv j v 2 fa, bg�g is also context-free because we canlet vR = w. Because the concatenation of two context-free languages is context-free,the resulting language which is formed this way is context-free.

C H A P T E R

Computability

5.1 SolutionsTHE STANDARD TURING MACHINE MODEL

5.1 Problem: Show that the standard Turing machine model of Section 5.1 and the modelof Section 3.7 are equivalent in that one can simulate the other.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: The next-state function of the Turing machine model of Section 5.1, Æ :Q � ( � [ f�g) 7! (Q [ fhg) � ( � [ f�g [ fL, Rg), maps the current state andvalue under the head into a next state and either a value to be written on the tapeor a head movement. However, the next state function for the model of Section 3.7,Æ : Q� (�[f�g) 7! (Q[fhg)� (�[f�g)�fL, N, Rg, differs in that it maps thecurrent state and value under the head into a next state and both a value to be writtenon the tape and a head movement.

Let the two models be Model5 and Model3, respectively. Model5 can be simulatedby Model3 as follows: To execute one step by Model5, execute one step by Model3 asfollows: a) if Model5 does not move the head, Model3 writes a potentially new tapesymbol without changing the head position, and b) if Model5 does move the head,Model3 writes the original value of the symbol in the current cell and performs thesame move as Model5.To simulate one step by Model3, Model5 performs one or two steps. On the first stepModel5 writes the symbol that Model3 would write. If on the current state and inputModel3 would move the head, Model5 executes a second step on which it moves the

91

92 Chapter 5 Computability

head. The second move, if needed, requires the addition of a new state for each stateand input on which Model3 moves the head.

PROGRAMMING THE TURING MACHINE

5.2 Problem: Describe a Turing machine that generates the binary strings in lexicograph-ical order. The first few strings in this ordering are 0, 1, 00, 01, 10, 11, 000, 001,: : : .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Such a Turing machine writes 0 on its otherwise blank work tape. Treatingthis string as a binary number, it increments it by adding 1 to the leftmost, least signif-icant position. If the new least significant bit is 1, it checks to see if all more remainingbits are also 1. If so, it replaces the blank to its right by 0 and replaces all other bits by0. It then returns to the first step which is to increment the binary number on its worktape by one.

5.3 Problem: Describe a Turing machine recognizing fxiyjxk j i, j, k � 1 and k = i �jg.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: We assume without loss of generality that the DTM for this problem hasa work tape. The DTM first verifies that the input string consists of x’s followed byy’s and then z’s. If it is not of this form, the string is rejected. To verify that thenumber of x’s, y’s, and z’s, namely i, j, and k, in xiyjxk satisfy k = i � j, the DTMuses markers on its input tape to copy xi onto the work tape for each instance of y inyj . The work tape now contains i � j instances of x. The DTM need only comparethese characters with the characters in the string zk. They have the same number ofcharacters if k = i � j.

5.4 Problem: Describe a Turing machine that computes the function whose value on inputaibj is ck, where k = i � j.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Given an input, the DTM first checks to see that it contains a string of a’sfollowed by b’s. It then invokes the procedure used by the DTM in the solution toProblem 5.3 to write a string of i � j characters on its work tape which can then betransferred to its output tape.

5.5 Problem: Describe a Turing machine that accepts the string (u,v) if u is a substringof v.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: On the input string (u,v), the DTM uses markers to determine if v con-tains u as a contiguous substring by trying to match each character in u starting withthe first position of v. (A marker can be viewed as a character on a second track. A sec-ond track can be simulated by using pairs of cells for each of the original cells, the firstcontains the tape character, the second contains the marker. A larger control unit canimplement this change.) If that fails, the DTM starts over trying to match u startingwith the second character of v.

5.1 Solutions 93

ControlUnit

Figure S.5.1 A schematic representation of a two-dimensional Turing machine.

5.6 Problem: The element distinctness language, Led, consists of binary strings notwo of which are the same; that is, Led = f2w12 : : :2wk2 j wi 2 B� and wi 6=wj, for i 6= jg. Describe a Turing machine that accepts this language.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: The DTM for this problem uses markers to compare w1 with each of theother strings. If it does not match these other strings, it repeats the process comparingthe next string with all subsequent strings until the next-to-last string has been com-pared with the last string. If no two strings match, the input is accepted; otherwise it isrejected.

EXTENSIONS TO THE STANDARD TURING MACHINE MODEL

5.7 Problem: Given a Turing machine with a double-ended tape, show how it can besimulated by one with a single-ended tape.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The double-ended tape is folded in half. The left half is placed on thebottom track of a single-ended two-track tape. The right half is placed on the toptrack. A special character is placed in the two tracks of the leftmost cell so that the TMcan detect that it has reached the left end of the single-ended tape. It then switchesdirection and moves to the second cell on the bottom track.

5.8 Problem: Show equivalence between the standard Turing machine and the one-tapedouble-headed Turing machine with two heads that can move independently on itsone tape.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The one-tape, single-ended, double-headed TM M2 can be simulated by theone-tape, single-ended, single-headed TM M1as follows: two special markers are used


by M1 to identify the positions of M2’s two heads. Then for each of the T2 steps ofM2, M1 sweeps over the non-blank portion of its tape from left to right to pick up thevalues under the two heads. It then sweeps back to the left end of the tape. When itreaches a marker for a head, it writes a potentially new symbol under the head and, ifnecessary, moves the head marker to an adjacent cell. It then resumes its sweep left untilit finds the next head marker or reaches the left-end of the tape. Thus, M1 will executeO(T 2

2 ) steps.

5.9 Problem: Show that a pushdown automaton with two pushdown tapes is equivalent toa Turing machine.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: With two pushdown stacks, a single, double-ended tape can be simulated.Since the input tape of a PDA is used as a one-way, read-only tape, the stacks are usedto simulate the two-way, read/write tape of a TM. We first describe how two stacks, S1

and S2, can be used to simulate a two-way, read/write tape. Let the value under thetape head be the element at the top of S1. To read it, it must be popped. If it is to bechanged, a new value is pushed. If the head is to be moved right, the element at the topof S2 is popped and then pushed onto S1. If the head is to be moved left, the elementat the top of S1 is popped and pushed onto S2. A special marker can be pushed ontoS1 to identify the left-most cell of a single-ended tape. The input can then be read fromthe read-only input tape and placed on the simulated tape after which the control unitmoves to the leftmost simulated cell and begins its simulation.

5.10 Problem: Figure 5.14 shows a representation of a Turing machine with a two-dimensionaltape whose head can move one step vertically or horizontally. Give a complete defini-tion of a two-dimensional TM and sketch a proof that it can be simulated by a standardTM.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: A formal definition of a 2-D Turing machine is given below.

DEFINITION 5.1.1 A two-dimensional Turing machine (TM) is a six-tuple M =(�,�,Q, Æ, s,h) where � is the tape alphabet not containing the blank symbol �,Q is the set of states, Æ : Q� ( �[ f�g) 7! (Q[ fhg)� ( �[ f�g [ fL, R, U, Dg)is the next-state function, s is the initial state, and h 62 Q is the accepting halt state.A TM cannot exit from h. If M is in state q with letter a under the tape head andÆ(q, a) = (q0, a0, C), its control unit enters state q0 and writes a0 if C = a0 2 � [ f�gor moves the head left (if possible), right, up, or down (if possible) if C is L, R, U, or D,respectively.

The TM M accepts the input string w 2 �� (it contains no blanks) if when startedin state s with w placed left-adjusted on its otherwise blank first row, the last state enteredby M is h. M accepts the language L(M ) consisting of all strings accepted by M .Languages accepted by Turing machines are called recursively enumerable. A language Lis decidable or recursive if there exists a TM M that halts on every input string, whetherin L or not, and accepts strings in L.

A function f : �� 7! �� [ ?, where ? is a symbol that is not in �, is partial if for somew 2 ��, f(w) =? (f is not defined on w). Otherwise, f is total.

5.1 Solutions 95

A TM M computes a function f : �� 7! �� [ ? for thosew such that f(w) is definedif when started in state s with w placed left-adjusted in the first row of its otherwise blanktape, M enters the accepting halt state h with f(w) written left-adjusted on in the first rowof its otherwise blank tape. If a TM halts on all inputs, it implements an algorithm. A taskdefined by a total function f is solvable if f has an algorithm and unsolvable otherwise.

To show that a 2-D TM can be simulated by a 1-D one, we give an ordering to thecells on the 2-D tape so that we can map each 2-D cell into a cell on the 1-D tape. Leteach cell be indexed by its row and column indices, r and c, where 1 � r, c. The cellin position (r, c) is given index p(r, c) = r + (r + c� 2)(r + c� 1)=2. This assignscell in the lower left corner the value 1, that to its right value 2, and that above it value3, etc. We simulate the 2-D TM with a 4-tape TM. Its first tape contains the entriesfrom the 2-D array. On its second and third tapes are kept the value of r and c. On thefourth tape the value p(r, c) is computed. Since this function involves integer additionand multiplication and division by 2, which is done by shifting, this computation istedious but straightforward. Once the value of p(r, c) is computed, it can be used tomove the tape head on the first tape from the leftmost cell to the cell numbered p(r, c).

The 1-D TM simulates the 2-D TM by determining which or r and c is to be incre-mented, incrementing it, computing p(r, c) and then advancing the head on the firsttape to the new position.

5.11 Problem: By analogy with the construction given in Section 3.9.7, show that everydeterministic T-step multi-tape Turing machine computation can be simulated on atwo-tape Turing machine in O(T logT ) steps.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: This solution is due to Hennie and Stearns, “Two-tape Simulation of Mul-titape Turing Machines,” JACM, vol. 13, no. 4, (1966), p. 533-546, and followsthe derivation given in the book “Introduction to Automata Theory, Languages, andComputation,” by Hopcroft and Ullman, Addison-Wesley, (1979).

Given a k-tape TM Mk that executes T steps, we simulate it by a 2-tape TM M2 thathas k pairs of tracks on its data tape, one pair for each of the k tapes, and two pairs oftracks on its work tape. The work tape is used only for the temporary storage of dataon the data tape.

M2 simulates the movements of Mk ’s k heads by moving data on the tracks of its datatape. After moving the data, k of the 2k tracks under M2’s head contain the data underMk ’s k heads. M2 reads these k values, makes the appropriate state transition, and thensimulates Mk ’s head movements by again moving on its data tape.

To arrange its data so that so that the data under its head is the same as under each ofMk ’s k heads, M2 moves data on one pair of tracks at a time. It moves data left on thetwo tracks associated with the jth tape of Mk’s tapes if the jth head moves right andmoves data right if jth head being simulated moves left. It is this movement of datathat causes the expansion of the execution time from T to O(T logT ).

Since the simulation of the k heads is done in succession, it suffices to describe M2’ssimulation of Mk’s first tape. The procedure employed assumes that the tape is doubleended (which is not essential) and rules the tape into blocksB0,B1,B�1,B2,B�2, : : :


x�7 x

�5 x�4 x�2 x0 x2 x4 x6x

�6 x�3 x

�1 x1 x3 x5 x7

B�3 B

�2 B2 B3B�1 B0 B1

Figure S.5.2 The layout of the blocks B0,B1,B�1,B2,B

�2, : : : on two tracks of M2’s datatape.

in which Bj contains 2j�1 contiguous tape cells each containing 2k tracks. (SeeFig. S.5.2 which shows the two tracks associated with Mk ’s first tape.)Before simulating the movement of Mk’s k heads, M2’s head is over block B0 whichcontains one cell. If Mk ’s first head moves left (right), the data on the first two tracksof M2’s data tape moves right (left). As the data is moved we show by induction thatthe following conditions apply to the blocks:

1. For each j > 0 one of the following holds: a) both tracks of Bj are full (containcharacters from Mk ’s first tape, some of which may be the blank �) and bothtracks of B�j are empty, b) both tracks of B�j are full and both tracks of Bj

are empty, or c) the bottom tracks of both blocks are full and their top tracks areempty.

2. A non-empty track of block B�j contains values from 2j�1 consecutive cells ofMk’s first tape. For j > 0, if both blocks of Bj are full, the upper track contains2j�1 entries to the immediate left of those on the lower track whereas if bothblocks of B�j are full, the upper track contains 2j�1 entries to the immediateright of those on the lower track.

3. For i < j the entries in Bi are to the left of those in Bj on Mk ’s first tape.

4. B0 has a special marker in its upper track and a tape value on its lower track.

The action of M2 when Mk moves the head on its first tape left is to move a blockof data right. (The action when Mk moves this head right is the mirror image of thisaction.) M2 searches right to find the first block, say, Bp, that does not have both tracksfull. (The actions that follow are called a Bp-operation.) Since B1,B2, : : : ,Bp�1 arefull, by the inductive hypotheses stated above, this implies that B�(p�1), : : : ,B�1 areempty. B0,B1,B2, : : : ,Bp�1 contain 1 + 2

�1 + 2 + � � �+ 2p�2

�= 2p � 1 entries.

Move these entries to the work tape and then to the lower track of B1,B2, : : : ,Bp�1

(erasing the other entries) and either the lower track of Bp if it is empty, or to Bp ’supper track otherwise. There are 1+2+22+ � � �+2p�1 = 2p�1 cells in these tracks,just enough to hold all the items that are being shifted right. (In the following explanaWe now describe the shifting of data on the data tape asM2 simulates the the movementof a head on Mk ’s tape. (See Fig. S.5.3 in which cell values are shown as they neverchange.)M2 moves left to B0, the marked cell, and then uses the 2p � 1 non-empty cells onthe work tape to find Bp on the data tape. (Blocks B�(p�1), : : : ,B�1 are empty.) Ifboth tracks of B�p are full, M2 copies the upper track to its work tape and then tothe lower tracks of B�(p�1), : : : ,B�1,B0. If both tracks are not full, M2 copies the

5.1 Solutions 97

B�3 B

�2 B2 B3B�1 B0 B1

x0

x0 x1

x0 x1

x0 x2x1 x3

x0 x2x1 x3

x�7 x

�5 x�4 x

�2 x0 x2 x4 x6x�6 x

�3 x�1 x1 x3 x5 x7

x�7 x

�5 x�4 x

�2 x�1 x2 x4 x6x

�6 x�3 x1 x3 x5 x7

x�7 x

�5 x�4 x

�2 x2 x4 x6x�6 x

�3 x�1 x3 x5 x7

x�7 x

�5 x�4 x

�3 x2 x4 x6x�6 x

�1 x3 x5 x7

x�6 x

�4 x�2 x4 x6x

�5 x�3 x

�1 x5 x7

x�6 x

�5 x�2 x4 x6x

�7 x�3 x

�1 x5 x7

x�2

x�7

x�4

Figure S.5.3 The movement of blocks as Mk ’s head moves left five times.

lower track of B�p to the lower tracks of these blocks. Since one track of B�p contains2p�1 cells as do the lower tracks of B�(p�1), : : : ,B�1,B0, there is just enough spacefor one track of B�p. This movement of data satisfies the conditions of the inductionhypotheses and places the contents of the cell to the left of B0 into the lower track ofB0.A Bp-operation, which takes K2p steps for some K > 0, is performed at most onceevery 2p�1 time steps because it takes this many steps to fill B1,B2, : : : ,Bp�1, whichare half empty after a Bp-operation. Thus, for at most once every T=2p�1 steps of Mk,1 � p � (log2 T ) + 1, M2 does a Bp-operation each costing K2p M2 steps. Thus,the total number of operations by M2, T2, satisfies:

T2 �log2 T+1Xp=1

K 2pT

2p�1

Thus, T2 = O (T logT ).

PHRASE-STRUCTURE LANGUAGES AND TURING MACHINES

5.12 Problem: Give a detailed design of a Turing machine recognizing fanbncn jn � 1g.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The table shown in Fig. S.5.4 describes a TM for this problem. It scans theinput sequence, rejecting it if it is not of the form aibjck and marking the first instanceof a, b, and c as a, b, and c, respectively. Except for the reject state r, the states areindexed by integers. The purpose of each state is explained below.


State a b c � a b c

1 2, a,R r,�,� r,�,� r,�,� r,�,� r,�,� r,�,�2 2, a,R 3, b,R r,�,� r,�,� r,�,� r,�,� r,�,�3 r,�,� 3, b,R 4, c,R ,�,� r,�,� r,�,� r,�,�4 r,�,� r,�,� 4, c,R 5,�,L r,�,� r,�,� r,�,�5 5, a,L 5, b,L 5, c,L r,�,� 6, a,R 5, b,L 5, c,L

6 7, a,R r,�,� r,�,� r,�,� r,�,� 10, b,R r,�,�7 7, a,R 8, b,R r,�,� r,�,� r,�,� 7, b,R r,�,�8 r,�,� 8, b,R 9, c,R r,�,� r,�,� r,�,� 8, c,R

9 r,�,� r,�,� 9, c,R 5,�,L r,�,� r,�,� r,�,�10 r,�,� r,�,� r,�,� h,�,� r,�,� 10, b,R 10, c,R

Figure S.5.4 The next-state table for the TM used in the solution of Prob 5.12.

State 1 is the initial state. The DTM M rejects the input unless the first letter is a whichit replaces with a after which it enters state 2. The purpose of this state is to advanceacross instances of a to the first instance of b which M replaces with b and enters state3. The purpose of state 3 is to move across b’s to the first c which it converts to c andenters state 4. In state 4 the DTM advances across c’s to the first blank at which pointit enters state 5. If in any of the above states a character is encountered that is out ofplace, the input is rejected. Thus, if the DTM enters state 5, the string on the tape isof the form aibjck.

In state 5, the DTM moves back across all characters until it reaches the rightmostinstance of a at which point it enters state 6 and moves right. If in state 6 it sees a, itreplaces it with a and enters state 7. If it sees b it knows that it has checked off all theinstances of a and it enters state 10 to see if it has also checked off all instances of b’s andc’s. In state 10 it moves right over instances of b and c, entering the halt state if it findsthe blank without encountering other characters, but rejecting the input otherwise. Instate 7 it moves across a’s and b’s until it finds the first b which it converts to b andenters state 8. If in state 7 it encounters c, it rejects the input because the input doesnot have as many b’s as a’s. In state 8 it moves across b’s and b’s until it reaches a c. If itfails to find a c, it rejects the input because it does not have equal numbers of the threecharacters. When it finds c in state 8 it converts it to c and enters state 9 during whichit moves right until it finds the first blank upon which it enters state 5.

5.13 Problem: Use the method of Theorem 5.4.1 to construct a phrase-structure grammargenerating fanbncn jn � 1g.

5.1 Solutions 99

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: This is a tedious exercise once the transition table for the solution has beenfound, as is done above. As shown in the solution to Problem 4.38, the context-freegrammar of Example 4.9.1 generates this language.

5.14 Problem: Design a Turing machine recognizing the language f02i j i � 1g.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: We scan the input to insure that is consists only of 0’s. We then use thegrammar given in the solution to Problem 4.39 to generate strings in this language. Ifwe generate a string that is shorter than the input, we generate the next string. If wegenerate one that is longer, we reject the input. See the solution to Problem 4.39 for adescription of the approach.

UNIVERSAL TURING MACHINES

5.15 Problem: Using the description of Section 5.5, give a complete description of a uni-versal Turing machine.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The universal TM (UTM) U executes five phases: a) it moves its head rightto find the marked input symbol; b) it searches left for the current state; c) it finds thevalue of the new tape symbol and the next state; d) it updates the current state; and e)it resumes phase a. Two versions of phase a are used, one to simulate the first step inthe simulation of a TM, and the second to simulate later steps. Let M be the TM thatis being simulated.

The leftmost instance of [ is marked, that is, is replaced byb[. This indicates that theinitial state is the first state. The leftmost symbol to the right of the rightmost triple< z0#q0 >< z00#q00 >< z00#q000 > is marked, that is, is either b0, b1, or b�. Thisindicates the input head of the simulated TM is initially over this cell.To find the marked input symbol (phase a) the UTM moves right from the leftmosttape cell and enters state qin. In this state, it moves right until the current symbol a isb0, b1, or b� whereapon it enters state qback,a.

States qback,0, qback,1, and qback,� are used to retain knowledge of the symbol that isunder the input head. While in one of these states the TM searches for the current state

(phase b) by moving left without changing the current state until it finds the symbolb[which is to the immediate left of the triple < z0#q0 >< z00#q00 >< z00#q000 > asso-ciated with the current state. While in qback,a it replacesb[ by [ and enters qforward,a .

When the UTM enters qforward,a for a 2 f0, 1,�g, it is entering phase c. In stateqforward,0 the UTM moves right on input < without changing its current state andreads its input, z0, and enters state qz0 . Its subsequent actions are described below.In state qforward,1 the UTM moves right without changing its state until it encounters> at which point it moves right and enters state qagain,1. It remains in this statemoving right until over z00 at which point it enters state qz00 . Its subsequent actions aredescribed below.In state qforward,� the UTM moves right without changing its state until it encounters> at which point it moves right and enters state qagain,�. It remains in this state moving


right until it is over > at which point it moves right and enters qencore,�. It remainsin this state moving right until over z000 on which it enters state qz000 . Its subsequentactions are described below.

After the TM enters state qz for z 2 f0, 1,�, L, Rg it moves to the immediate right of#. If it finds it finds >, M has entered its halt state. Thus, the UTM U enters its ownhalt state, qhalt. If the character to the right of # is 1, U replaces it by b1 and entersstate qz,left. It is now entering phase d during which U changes the state of M .

The UTM U enters state qz,left so that it can mark the triple associated with the next

state of the simulated TM by changing its [ tob[. In state qz,left the UTM moves to theleftmost tape cell. It does this by moving left until it finds a special symbol . markingthe left-hand end of the tape and then moves right. It replaces [ byb[ and enters stateqz,search in which it moves right until it finds b1. At this point it replaces b1 by 1 andenters state qz,first. In this state it moves right and enters state qz,second. If in qz,second

the tape contains >, the next state is properly marked which means that it can returnto phase a. If not, it must move the mark on a [ right at least one more place.

In the first case, namely, when the triple corresponding to the next state has beenmarked, the UTM moves right and enters state qz,in in which it continues to moveright until it finds either b0 or b1. In this second version of phase a the knowledge ofz is carried forward by the UTM. Its action is determined by z which is 0, 1, �, Lor R. In the first three cases it replaces the value in the cell by the appropriate markedinput symbol and enters the appropriate state qback,0, qback,1, and qback,� which retainsknowledge of the value under the head. In the last two cases, it enters states in whichit unmarks the current symbol, moves to the appropriate adjacent cell, and marks itsvalue. It then enters the appropriate state qback,0, qback,1, and qback,�.

However, if in qz,second the tape contains 1, the UTM replaces it by b1 and entersqz,third. In this state it moves left and enters state qz,left in which it moves left until

it findsb[ in which case the UTM replaces it by [ and enters qz,fourth. In qz,fourth the

UTM moves right until it finds [ which it replaces withb[ and enters qz,search to resumephase b. (Note that no instances ofb[ exist on the first step of phase b.)

5.16 Problem: Construct a universal TM that has only two non-accepting states.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: This solution is due to Claude Shannon, “A Universal Turing Machine withTwo Internal States,” Automata Studies, J. McCarthy and C. E. Shannon, Eds., Prince-ton University Press, (1965), p. 157-165. This universal machine is designed for aTM model similar to that of Chapter 3 in which the next-state function maps thecurrent state and value under the head to a next state, a new value under the head,and a left or right head movement command. That is, the tape head is not allowedto stay on the current cell. For this model the next-state function has the followingdomain and range: Æ : Q � (� [ f�g) 7! (Q [ fhg) � (� [ f�g) � fL, Rg.More than two states may be necessary for the model of this chapter, namely, whenÆ : Q� ( � [ f�g) 7! (Q [ fhg)� ( � [ f�g [ fL, Rg).We design a two-state TM TWO with state set Q(2) and tape alphabet �(2) that sim-ulates an arbitrary TM M with state set Q and tape alphabet �. TWO uses multiplesteps to simulate one step by M . If M moves from cell i to cell i + 1 (i � 1), TWO

5.1 Solutions 101

bounces between cells i and i + 1 (i � 1) multiple times to carry M ’s state from cell ito cell i+ 1 (i� 1) as well as record the new value that is printed in cell i by M beforeit makes the transition.Let Q = fq1, q2, : : : , qKg and Q(2) = fq(2)0 , q(2)1 g. If � = f 1, 2, : : : , Jg, thealphabet for TWO is the following.

�(2) = � [ f k,j,t,b j 1 � k � K, 1 � j � J , t 2 f+,�g, b 2 fL,Rgg

The letters k,j,�,b and k,j,+,b are used to convey information between adjacent cellswhen bouncing between them. Here j indicates that j is to remain in the current cellafter the bouncing is done and k is the index of the current state of M . k,j,�,b indicatesthat the cell containing this letter is receiving information during the bouncing phasewhereas k,j,+,b indicates that the cell is supplying information. The value of b specifiesthe direction in which the head is to move on the next bounce.The inductive hypothesis has the following two parts: a) before TWO starts the simula-tion of the next step of M when M ’s head is in state qk with its head over cell i whichcontains j , TWO is in state q(2)0 and its head is over cell i which contains k,j,�,m form 2 fL,Rg, and b) all cells other than the cell over which TWO’s head resides containthe same letters from M ’s alphabet � as does M .If the transition by M in state qk on input j is Æ(qk, j) = (qk0, j0 ,m) for m 2fL,Rg, we let TWO’s next-state function Æ(2) satisfy

Æ(2)(q(2)0 , k,j,�,x) = (q, k0 ,j0,+,m,m)

where q = q(2)0 if m = L and q = q

(2)1 if m = R, independently of the value of

x. Thus, TWO writes k0,j0,+,m in cell i, changes to a state associated with the headmovement by M and moves its head to the cell to which M moves its head.

After this transition TWO has entered the bouncing phase in state q(2)0 if it has movedto the current state from the cell to its right and in state q(2)1 otherwise. The old cellcontains the letter k0,j0,+,m whereas the new one contains some letter u.We let TWO have the following additional next-state transitions:

Æ(2)(q(2)0 , u) = (q

(2)0 , 1,u,�,R,R)

Æ(2)(q(2)1 , u) = (q

(2)0 , 1,u,�,L,L)

Thus, if TWO is in state q(2)0 when it enters a cell containing u, it writes 1,u,�,R toindicate that it has entered this cell from the cell on its right and it moves right to this

old cell. If under the same conditions it is in state q(2)1 , it writes 1,u,�,L to indicatethat it has entered this cell from the cell on its left and moves left to this old cell. Thethird subscript,�, indicates that the cell containing this letter is a receiving cell. It willreceive M ’s state from the old cell.Summarizing, after two moves TWO’s head is over the old cell which contains k0 ,j0,+,m

and the new cell contains 1,u,�,b where b is the complement of m (b = R if m = L

and vice versa). TWO now bounces back and forth between the two cells, decrementingthe first subscript of the letter in the old cell by 1 as it increments the first subscript of


the letter in the new cell. TWO is in state q(2)0 when over the new cell and in state q(2)1when over the old cell until the first subscript of its letter decreases to 1. At this pointTWO changes the letter in the old cell to j0 and moves to the new cell in state q(2)0 .This has the effect of restoring the conditions of the inductive hypothesis that applybefore simulating each step of M .To cause the above actions to occur during bouncing, we add the following next-state

transitions for 1 � k � K, 1 � j � J , m 2 fL,Rg, and q 2 fq(2)0 , q(2)1 g:

Æ(2)(q(2)1 , k,j,�,m) = (q

(2)0 , k+1,j,�,m,m)

Æ(2)(q, k,j,+,m) = (q(2)1 , k�1,j,+,m,m)

Æ(2)(q, 1,j,+,m) = (q(2)

0 , j ,m)

The first type of transition causes k to increment whereas the second causes it to decre-ment. Thus, as TWO bounces between the old and new cells it decrements the stateindex in the old cell and increments it in the new one until the third type of rule isinvoked and the new value is written into the old cell and TWO moves to the new cell.

DECIDABLE PROBLEMS

5.17 Problem: Show that the following languages are decidable:

a) L = f�(M ),w j M is a DFSM that accepts the input string wgb) L = f�(M ) jM is a DFSM and L(M ) is infiniteg This problem has changed.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution:

a) �(M ),w is in L if the DFSM M accepts input w. To test if M accepts w weconstruct a TM M 0 that simulates M on w. This simulation terminates on allinputs because M is a DFSM. If the simulation ends in an accepting state then M 0

accepts. Otherwise it rejects.b) From Lemma 4.5.2 the language accepted by M is infinite if and only if it contains

a string of length at least m and at most 2m � 1 where m is the number of statesof M . We merely have to simulate M on all strings whose length is in this rangeto see if they are accepted.

5.18 Problem: The symmetric difference between sets A and B is defined by (A � B) [(B � A), where A � B = A \ B. Use the symmetric difference to show that thefollowing language is decidable:

LEQ FSM = f�(M1), �(M2) jM1 and M2 are FSMs recognizing the same languageg

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Solution: If the languages recognized by the FSMs M1 and M2, LM1 and LM2 , aredifferent then either LM1 � LM2 or LM2 � LM1 is not empty. Thus, they recog-nize the same languages if and only if L0 = (LM1 � LM2) [ (LM1 � LM2) is notempty. Since LM1 and LM2 are regular and the set of regular languages are closed un-der complements, intersections, and unions, L0 is regular. Thus, we can use the resultof Problem 5.17 to show that the problem posed in this question is decidable.

5.1 Solutions 103

5.19 Problem: Show that the following language is decidable:

L = f�(G),w j �(G) encodes a CFG G that generates wg

Hint: How long is a derivation of w if G is in Chomsky normal form?

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: We assume that G is in Chomsky normal form. We know that some deriva-tion tree for the terminal string w has depth at most log2 jwj. We design a TM M

that systematically produces all derivations of G of this length or less. If it produces w,the input is accepted. Otherwise, it is rejected.

A procedure that this TM can use to systematically generate derivations is based on thefollowing tree model for derivations: a) the root of the tree is associated with the startsymbol; b) the root has an edge to one subtree for each derivation that has the start sym-bol on the left-hand side; c) each of these subtrees has either one descendant, a vertexassociated with a terminal, or two descendants associated with the non-terminals on theright-hand side of its derivation; d) when there are two descendants, the two verticeshave one subtree for each derivation that has the corresponding non-terminal on theleft-hand side; e) this process is repeated until with each descendant until the appropri-ate depth is reached. If the rules with a common left-hand side are numbered, they canbe used in this ordering to generate strings systematically in a depth-first fashion untilthe appropriate depth is reached.

5.20 Problem: Show that the following language is decidable:

L = f�(G) j �(G) encodes a CFG G for which L(G) 6= ;g

This problem has changed.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: We design a TM M 0 that when given as input a description of a CFG G firstmarks all the terminals of the grammar and it then scans all the rules of the grammarand marks non-terminal symbols that can be replaced by some marked symbols (so ifthere is a non-terminal A that is not marked and a rule A ! BCD where B,C,Dhave already been marked, then it also marks A). We repeat this procedure until nonew non-terminals can be marked. Then if S is marked, we accept; otherwise we rejectbecause it is not possible to generate a string of terminals from S.

5.21 Problem: Let L1,L2 2 P where P is the class of polynomial-time problems (see Defi-nition 3.7.2). Show that the following statements hold:

a) L1 [ L2 2 P

b) L1L2 2 P, where L1L2 is the concatenation of L1 and L2

c) L1 2 P


. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) L1 [L2 can be recognized in timeO(P1(n)+P2(n)) by runningM1 and M2 onthe same input and accepting if at least one of them accepts.

b) Membership in L1L2 can be decided in time O((n + 1)�P1(n) + P2(n)

�) by

taking an input string w, forming all n + 1 partitions of w (n = jwj) into twosubstrings w1,w2, and checking for each partition whether w1 belongs to L1 andw2 belongs to L2.

c) Just invert the answer of M1, at a small constant cost. The running time isO(P1(n)).

5.22 Problem: Let L1 2 P. Show that L�1 2 P.

Hint: Try using dynamic programming, the algorithmic concept illustrated by theparsing algorithm of Theorem 4.11.2.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Membership in L�1 can be decided in time O(n3P1(n)) using the followingdynamic programming approach (the algorithm is very similar to that of CYK). For aninput string w 6= e of length n and 1 � i � j � n, let wij be the substring extendingfrom the ith to the jth character of w; and let xij be a Boolean value indicating whetherwij is a member of L

�

1 or not. The problem is then equivalent to determining the truthvalue of x1n. We have the following recursion:

xij = (wij 2 L1) __

i�k<j

(xik ^ xk+1,j),

which says that wij is a member of L�

1 either if it is a member of L1 or if it can bebroken into two pieces each belonging to L

�

1 . Proceeding bottom-up from shorterto longer substrings of w, we can determine the value of each xij by checking wij formembership inL1 and then looking at the 2(j�i) values xik andxk+1,j for i � k < j.This can be done in O(nP1(n) + n) steps. Since there areO(n2) substrings of w, x1n

and hence the result can be computed in O(n3(P1(n) +n)) time. The empty string isalways accepted.

UNSOLVABLE PROBLEMS

5.23 Problem: Show that the problem of determining whether an arbitrary TM startingwith a blank tape will ever halt is unsolvable.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: We assume there is such a halting TM ME for this problem and show it canbe used to solve the halting problem as follows: Given an encoding �(M ) of a TM M

and an input w to this machine, we construct a TM MS that examines its tape. If itis empty, MS writes w and then invokes a universal TM to simulate M on w whichhalts if and only if M on w halts. As a consequence, the halting problem for TM M

on input w can be solved by packaging up M and w as MS and then giving it withthe empty tape to ME . ME halts if and only if M on input w halts. Thus, we have asolution to the halting problem, which we know does not exist.

5.1 Solutions 105

5.24 Problem: Show that the following language is undecidable:

LEQ = f�(M1), �(M2) j L(M1) = L(M2)g

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: We assume there is a TM M that solves this problem and show that a con-tradiction results. To determine if M1 accepts the empty tape, we construct a simpleTM M2 that does not accept any input. We then pass �(M1), �(M2) to M which de-termines if L(M1) is empty, thereby recognizing the language of Problem 5.23 whichwe know is not solvable.

5.25 Problem: Determine which of the following problems are solvable and unsolvable.Defend your conclusions.

a) f�(M ),w, p jM reaches state p on input w from its initial stategb) f�(M ),p j there is a configuration [u1 : : : umqv1 : : : vn] yielding a configuration

containing state pgc) f�(M ), a jM writes character a when started on the empty tapegd) f�(M ) jM writes a non-blank character when started on the empty tapege) f�(M ),w j on input wM moves its head to the leftg. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

a) Undecidable. Reduction from the halting problem. If this problem were decidable,given a TM M and a stringw we could use it to decide if M halts on w by testingif M enters state h on input w.

b) Decidable. It suffices to check the transition function of M for a state q 6= p suchthat Æ(q, v1) = (p,�0), for some v1 2 � and �0 2 � [ fL,Rg.

c) Undecidable. Modify M such that it prints a special symbol $ just before enteringthe halting state. If we could decide whether the modified machine ever prints a $sign, we could decide whether M halts on the empty tape, a problem that is easyto show unsolvable: given a machine M and an input w construct a machine M 0

that first writes w on the tape and then simulates M . M 0 halts on the empty tapeif and only if M halts on w.

d) Decidable. Simulate M until either (1) M prints a nonblank symbol, (2) M halts,or (3) M enters a state it has reached before (in which case M will loop forever,since it is deterministic and its input continues to be blank). Answer “yes” in thefirst case and “no” in the other cases. Note that one of these cases must occur afterat most jQj steps, where Q is the set of states.

e) Decidable. Simulate M until either (1) M moves its head left, (2) M halts, or(3) M enters a state/current input symbol combination it has reached before (inwhich case an infinite loop is present, since the future computation of M depends,in the absence of left moves, only on the current state and input symbol). Theanswer is “yes” in the first case and “no” otherwise; and it will be given after atmost jQj � j�j steps.


FUNCTIONS COMPUTED BY TURING MACHINES

5.26 Problem: Define the integer division function fdiv :N2 7!N using primitive recur-sion.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The value of fdiv(x, y) is bx=yc. To compute it we first define an auxil-iary function fquot rem(x, y, q, d) that computes dx=ye. We then use this function tocompute bx=yc.The function fquot rem(x, y, q, d) carries not only the value of x and y but also aquotient q and a difference d. Here x = q � y + d or d = x � q � y. Instead ofcomputing d this way, we let d = x � q � y where � is proper subtraction. Thus,d = x when q = 0 and d = 0 when q is dx=ye. Hence, we define fquot rem(x, y, q, d)as follows:

fquot rem(x, y, q, 0) = q

fquot rem(x, y, q, d) = fquot rem(x, y, q + 1, d�y)

We let fdx=ye(x, y) = fquot rem(x, y, 0,x). Thus, on each invocation of the secondversion of fquot rem(x, y, q, d) the value of d is decreased by y and q is increased by 1until d is 0 at which point the resulting value is the value of dx=ye.To compute bx=yc we use the function fif�then�else(p, a, b) that has value a if p � 1and value b if p = 0. That is, fdiv(x, y) = fif�then�else(p, a, b) where p = dx=ye �y�x, a = dx=ye, b = P (dx=ye), and P (x) is the predecessor function. The functionfif�then�else(p, a, b) is defined recursively below.

fif�then�else(0, a, b) = a

fif�then�else(1, a, b) = b

fif�then�else(p, a, b) = fif�then�else(p� 1, a, b)

5.27 Problem: Show that the function fremain : N2 7! N that provides the remainder ofx after division by y is a primitive recursive function.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: fremain(x, y) = fsub(x, fmult(x, fdiv(x, y)))

5.28 Problem: Show that the factorial function x! is primitive recursive.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution:

ffact(1) = 1

ffact(x) = fmult(x, ffact(x� 1))

5.29 Problem: Write a RAM program (see Section 3.4.3) to realize the composition opera-tion.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The composition of g : Nm 7! N of m arguments with m functionsf1 :Nn 7!N, f2 :Nn 7!N, : : : , fm :Nn 7!N, each of n arguments, is defined

5.1 Solutions 107

by the following:

f(x1,x2, : : : ,xn) = g(f1(x1,x2, : : : ,xn), : : : , fm(x1,x2, : : : ,xn))

Let Pg and Pf1 ,Pf2 , : : : Pfm be RAM programs that compute g and f1, f2, : : : , fm ,respectively, and let R1, : : : , Rk be the maximal set of registers which any of them uses.Assume that these programs leave the values of their corresponding functions in registerR1. Let the values of x1,x2, : : : ,xn be stored in registers R1, : : : , Rn. A RAM programfor this composition copies the contents of R1, : : : , Rn into Rk+1, : : : , Rk+n and thenuses Pf1 to compute the value of f1(x1,x2, : : : ,xn) on R1, : : : , Rn. The value off1(x1,x2, : : : ,xn) is then stored in register Rk+n+1, the values of R1, : : : , Rn arerestored and the values of registers Rn+1, : : : , Rk are cleared (restored to their originalvalues). This process is repeated for fj(x1,x2, : : : ,xn) for j = 2, 3, : : : ,m exceptthat the value of fj(x1,x2, : : : ,xn) is stored in register Rk+n+j. After computingfm(x1,x2, : : : ,xn) and storing its value, the contents of fj(x1,x2, : : : ,xn) is storedin register Rj for 1 � j � m and registers Rm+1, : : : , Rk cleared. The program forPg is then invoked to compute g(f1(x1,x2, : : : ,xn), : : : , fm(x1,x2, : : : ,xn)). Theprogram for this task is shown in Figure 5.5.

5.30 Problem: Write a RAM program (see Section 3.4.3) to realize the primitive recursionoperation.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: The program shown in Fig. S.5.6 implements primitive recursion. It isassumed that the functions g and h use only registers Rj for 1 � j � k and thatthe n + 1 arguments to f(x1,x2, : : : ,xn, y) are located initially in registers R1, R2,: : : , Rn+1, respectively. The program computes f(x1,x2, : : : ,xn, y) by computingf(x1,x2, : : : ,xn, 0), f(x1,x2, : : : ,xn, 1), etc. until reaching f(x1,x2, : : : ,xn, y).The program begins by moving the n + 1 arguments to higher indexed registers. Theinitial value of y, which appears in Rn+1, is copied to Rk+n+1 so that it is available tobe decremented until it reaches 0. The program clears registers Rj for n + 1 � j � k

and executes the program Pg to compute f(x1,x2, : : : ,xn, 0). Its value, which isassumed to reside in R1, is moved to Rk+n+3. Register Rk+n+2 contains 0 initially andis incremented each time that Rk+n+1 is decremented.The program then tests to see if the iteration number Rk+n+1 has been reduced to0 (and Rk+n+2 = z has been increased to y). If so, the program terminates; if not,the arguments x1,x2, : : : ,xn are restored, the current value for z (Rk+n+2) is putinto Rn+1 and the value for f(x1,x2, : : : ,xn, z) placed into Rn+2. The program forh, Ph, is then executed on these arguments to compute f(x1,x2, : : : ,xn, z + 1) =h(x1,x2, : : : ,xn, z, f(x1,x2, : : : ,xn, z)).

5.31 Problem: Write a RAM program (see Section 3.4.3) to realize the minimalization op-eration.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solution: Minimilization is defined in terms of functions f : Nn 7! N and g :N

n+1 7! N. Here f(x) be the smallest integer y 2 N such that g(x, y) = 0 andg(x, z) is defined for all z � y, z 2N.We assume that only registers Rj, 1 � j � k, are used by the program for g and thatits value is left in R1. The value of f is left in R1 also.


The program given in Fig. S.5.7 saves the arguments x1,x2, : : : ,xn and clears thesearch variable y which is held in Rk+n+1. It then restores the arguments and loads thenext larger value of y into Rn+1. It clears registers Rj, n + 2 � j � k, and executesthe program Pg for g. If the result is 0, the program for f terminates after loading thevalue of y into R1. Otherwise, it increments y and repeats the execution. We assumethat the labels N0 and N1 are not used in Pg.

5.1 Solutions 109

f g(f1, : : : , fm) CommentsN0 Rk+1 R1 Save arguments

:::

Rk+n Rn

Pf1 Compute f1(x1,x2, : : : ,xn)Rk+n+1 R1 Save resultR1 Rk+1 Restore arguments:::

Rn Rk+n

CLR Rn+1 Clear working registers:::

CLR Rk

Pf2 Compute f2(x1,x2, : : : ,xn)Rk+n+2 R1 Save resultR1 Rk+1 Restore arguments:::

Rn Rk+n

CLR Rn+1 Clear working registers:::

CLR Rk

:::

Pfm Compute fm(x1,x2, : : : ,xn)Rk+n+m R1 Save resultR1 Rk+n+1 Retrieve values of functions:::

Rm Rk+n+m

CLR Rm+1 Clear working registers:::

CLR Rk

Pg Compute g(f1(x1,x2, : : : ,xn), : : : , fm(x1,x2, : : : ,xn))CONTINUE

Figure S.5.5 A RAM program to implement functional composition.


Primitive Recursion CommentsRk+1 R1 Save arguments:::

Rk+n Rn

Rk+n+1 Rn+1 Save y, number of iterations neededCLR Rk+n+2 Initialize z, current number of iterations, to 0CLR Rn+1 Clear working registers:::

CLR Rk

Pg Compute f1(x1,x2, : : : ,xn, 0) = g(x1,x2, : : : ,xn)Rk+n+3 R1 Save result

N0 Rk+n+1 JMP� N1 Exit if doneR1 Rk+1 Restore arguments:::

Rn Rk+n

Rn+1 Rk+n+2 Use current iteration number, zRn+2 Rk+n+3 Use value of f1(x1,x2, : : : ,xn, z)CLR Rn+3 Clear working registers:::

CLR Rk

Ph Compute h(x1,x2, : : : ,xn, z, f1(x1,x2, : : : ,xn, z))Rk+n+3 R1 Save resultDEC Rk+n+1 Decrement yINC Rk+n+2 Increment zJMP+ N0 Jump back to test for another iteration

N1 CONTINUE

Figure S.5.6 A RAM program to implement primitive recursion.

5.1 Solutions 111

Minimilization CommentsRk+1 R1 Save arguments:::

Rk+n Rn

CLR Rk+n+1 Initialize the search variable y to 0N0 R1 Rk+1 Restore arguments

:::

Rn Rk+n

Rn+1 Rk+n+1 Load current value of yCLR Rn+2 Clear working registers:::

CLR Rk

Pg Compute g(x1,x2, : : : ,xn, y)R1 JMP� N1 End computation if R1 is 0INC Rk+n+1 Increment yJMP+ N0 Redo the computation with next larger value of y

N1 R1 Rk+n+1 Load value of f(x1,x2, : : : ,xn)CONTINUE

Figure S.5.7 A RAM program to implement minimalization.

instructor’s manual volume i models of computation

Documents