1 dsa seminar – march 10

T.U.C

luj–M

. Jold

os

1 DSA seminar – March 10


Ex. 1.1. A linked list has exactlyn nodes. The elements in these nodes are selected from the set0, 1, ..., n. There are noduplicates in the list. Design anO(n) worst case time algorithm to find which one of the elements from the aboveset is missing in the given linked list.

Solution. Note there aren + 1 numbers and onlyn are on the list. It is known that the sum∑n

i=0 i =∑n

i=1 i =n(n+1)

2 . The idea is to addi to a counter initialized to 0. To get the missing number you should subtract the sum

sum of list elements from the sum given by the formulan(n+1)2 . Examples:

List contents Number of elements (n) List sum Formula sum Missing number〈0, 1, 2, 3〉 4 6 10 4〈1, 2, 3, 4〉 4 10 10 0〈0, 1, 3, 4〉 4 8 10 2

Ex. 1.2. LetA be an array of sizen, containing positive or negative integers, withA[1] < A[2] < . . . < A[n]. Design anefficient algorithm (should be more efficient thanO(n)) to find ani such thatA[i] = i provided such ani exists.What is the worst case computational complexity of your algorithm ?

Solution. Should use binary search and checkA[1] first.Ex. 1.3. A queueQ contains the itemsa1, a2, . . . , an, in that order, witha1 at the front andan at the rear. It is required to

transfer these items on to a stackS (initially empty) so thata1 is at the top of the stack and the order of all otheritems is preserved. Usingenqueue anddequeue operations for the queue andpush andpop operations for thestack, outline an efficientO(n) algorithm to accomplish the above task, using only a constant amount of additionalstorage.

Solution. Dequeuing all elements and pushing them onto the stack will result in a stack withan at the top anda1 atthe bottom. This is done inO(n) time asdequeue andpush each require constant time per operation. The queueis now empty. By popping all elements and pushing them on the the queue we will geta1 at the top of the stack.This is done again inO(n) time. As in big-oh arithmetic we can ignore constant factors, the process is carried outin O(n) time. The amount of additional storage needed here has to be big enough to temporarily hold one item.

Ex. 1.4. A queue is set up in a circular arrayC[0..n− 1] with front andrear defined as usual. Assume thatn− 1 locationsin the array are available for storing the elements (with theother element being used to detect full/empty condition).Derive a formula for the number of elements in the queue in terms ofrear, front, andn.

Solution. With this kind of queue (see Figure 1) we have:

• Rear of the queue is somewhere clockwise from thefront• To enqueue an element, we move rear one position

clockwise and write the element in that position• To dequeue, we simply move front one position

clockwise• Queue migrates in a clockwise direction as we en-

queue and dequeue• Emptiness and fullness to be checked carefully.

Analyze the possible situations (make some drawings tosee wherefront andrear are when the queue is empty,and partially and totally filled). You will get this:

NumberOfElements =

rear − front + 1 if rear ≥ frontrear − front + n otherwise

Rear 0

0 Front

0

MaxLength-1

1

Figure 1: Circular array implementation of a queue.Note that the grey part is filled.

Ex. 1.5. Write an iterativeReverse()∗ function that reverses a list by rearranging all the.next pointers and the headpointer. Ideally,Reverse() should only need to make one pass of the list. The iterative solution is moderatelycomplex.

Solution. This first solution uses the "Push" strategy with the pointerre-arrangement hand coded inside the loop.

∗from Nick Parlante’sLinked List Problemshttp://cslibrary.stanford.edu/105/LinkedListProblems.pdf

DSAL.105.AC.01.04-B2 1

http://cslibrary.stanford.edu/105/LinkedListProblems.pdf

T.U.C

luj–M

. Jold

os


There’s a slight trickyness in that it needs to save the valueof thecurrent->next pointer at the top of the loopsince the body of the loop overwrites that pointer. Assume that the data structure for a node is:

struct node

int data ;struct node∗ next ;

;

and the list has no header cell.

/∗I terat ive l i s t reverse .I terate through the l i s t le f t−r ight .Move/ inser t each node to the front of the resul t l i s t−−

l i ke a Push of the node .∗/static void Reverse(struct node∗∗ headRef)

struct node∗ result = NULL;struct node∗ current = ∗headRef;struct node∗ next ;while (current != NULL)

next = current−>next ; / / t r icky : note the next nodecurrent−>next = result ; / / move the node onto the resul tresult = current ;current = next ;

∗headRef = result ;

Another solution is:

/ / Reverses the given linked l i s t by changing i t s . next pointers and/ / i t s head pointer . Takes a pointer ( reference ) to the head pointer .void Reverse(struct node∗∗ headRef)

i f (∗headRef != NULL) / / special case : skip the empty l i s t

/∗Plan for th is loop : move three pointers : front , middle , backdown the l i s t in order . Middle is the main pointer runningdown the l i s t . Front leads i t and Back t ra i l s i t .For each step , reverse the middle pointer and then advance allthree to get the next node .∗/struct node∗ middle = ∗headRef; / / the main pointerstruct node∗ front = middle−>next ; / / the two other pointers (NULL ok)struct node∗ back = NULL;while (1)

middle−>next = back ; / / f i x the middle nodei f ( front == NULL) break ; / / tes t i f doneback = middle; / / advance the three pointersmiddle = front ;front = front−>next ;

∗headRef = middle; / / f i x the head pointer to point to the new front

Another approach is to use recursiveness. Probably the hardest part is accepting the concept that the procedureRecursiveReverse(&rest) does in fact reverse the rest. Then then there’s a trick to getting the one frontnode all the way to the end of the list. Make a drawing to see howthe trick works.

void RecursiveReverse (struct node∗∗ headRef)

struct node∗ f i r s t ;

DSAL.105.AC.01.04-B2 2

T.U.C

luj–M

. Jold

os


struct node∗ rest ;i f (∗headRef == NULL) return ; / / empty l i s t base casef i r s t = ∗headRef; / / suppose f i r s t = 1 , 2 , 3rest = f i r s t−>next ; / / rest = 2 , 3i f ( rest == NULL) return ; / / empty rest base caseRecursiveReverse(& rest ) ; / / Recursively reverse the smaller 2 , 3 case/ / af ter : rest = 3 , 2f i r s t−>next−>next = f i r s t ; / / put the f i r s t elem on the end of the l i s tf i r s t−>next = NULL; / / ( t r icky step−− make a drawing)∗headRef = rest ; / / f i x the head pointer

Ex. 1.6. Consider the following recursive definition:

g(i, j) = i if j = 1g(i, j) = j if i = 1g(i, j) = g(i− 1, j) + g(i, j − 1) otherwise

Design an iterative algorithm running inO(mn) time to computeg(m, n), where bothm andn are positive integers.

Solution. Use two embeddedfor loops with the loop control variables taking values from 1 tom, andn, respectively.Ex. 1.7. Problem 1.6 from Laboratory guide. The elements of asetM are denoted by lowercase letters. Read pairs of

elements(x, y), x, y ∈ M meaning "elementx precedes elementy. List the elements of the set such that if anelement precedes another, then it should be listedbeforeits successors. Example input:

a,da,ee,df,ec,f

Solution. Maintain for each element a list of mandatorypreceding elements for it. For the given set, one possi-ble arrangement in the list is given if Figure 2. Thereare four possible cases for e given pair:

a) None of the elements are in the list. Insert sec-ond at the front of the top list, then the firts in itsprecedence list, then insert the first at the front ofthe top list.

b) First is on list. Insert second element after firstin the top list, and also insert the first in second’sprecedence list.

c) Second on list, first not on list. Insert first at thefront on the top list, then add it to second’s prece-dence list.

d) Both on list. This case is a little bit tricky. If firstprecedes second in top list, then just add first tosecond’s precedence list (if not already there). Iffirst is after second in top list, then check howfar left you can promote second by checking itsprecedence list. If you cannot promote it beforefirst, issue an error message.

a—•

c—•

f— e— d— •

c

•

c

•

c

•

c c

c

•

c

•

c

•

f f

c

•

c

•

c

•

a

•

e

c

•

c

•

c

•

c

•

a

•

Figure 2: Example of lists for Ex. 1.7.

DSAL.105.AC.01.04-B2 3

T.U.C

luj–M

. Jold

os



Ex. 2.1. Sketch a non-recursive preorder traversal of a binary treeT . How would you change your code to work on aleftmost child – right siblinggeneral tree representation?

Solution. Preorder can be implemented using a stack, without recursion. Here is a pseudocode for binary treepreorder.

void PreorderTraversal(Tree T)

Stack S;

S = createEmpty; // initialize stackpush(root(T), S); // push root node onto stack Swhile (!isEmpty(S))v = pop(S); // here we assume that pos also retrieves top element of Sif (v != NULL)

visit v // simplest case: visit = just list itpush(v’s right child, S);push(v’s left child, S); // question: why right child pushed first?

Note that for the general tree, instead of only one right child we have all the right siblings of a node. One idea isthe build an equivalent binary tree for the general tree – in the equivalent binary tree the list of siblings is alwaysrepresented by the right branch, and the leftmost child by a left branch (see Figure 3)

(a) A tree (b) Leftmost child–right sibling (c) Binary tree

R

A

C D E

B

F

R •

A B •

C • D • E • F • •

R •

A

C • B •

D • F • •

E •

Figure 3: A tree and two possible representations for it.

Ex. 2.2. Suppose you are given the preorder and postorder traversals of a binary tree. Can you reconstruct the tree? If so,give an algorithm for doing so. If not, give a counter example.

Solution. No. If we have left only or right only subtrees in the treethen the preorder and postorder traversals are not necessarily unique.See the following example. Both the trees have preorder ( 1 2 3) andpostorder ( 3 2 1 ).

1

2

3

1

2

3

Figure 4: Binary trees.

Ex. 2.3. The following are the inorder and postorder traversals of a single binary tree whose 10 nodes are labeled 0, 1, 2,. . ., 9 .

inorder: 4 1 5 6 2 0 8 3 9 7postorder: 4 6 5 2 1 8 9 7 3 0

a) Draw the corresponding treeT with the nodes labeled.

DSAL.105.AC.01.04-B2 4

T.U.C

luj–M

. Jold

os


b) Design an algorithm for constructing trees from in– and post– order traversals of a binary tree

Solution. We know that the root is:

• Thefirst element in apreordertraversal.• Thelast element in apostordertraversal.

Let then the position of the root in the inorder traversal bei. The string characters from the first element toi − 1represent the inorder traversal of the left subtree, and thestring characters beyond theitth element to the end of thestring represent the inorder traversal of the right subtree. As the number of characters in any traversal listing is thesame, we ca easily split the inorder traversal into left and right subtrees, recursively.

Here is a routine that reconstructs a binary tree (note that it works if nodes are identified by single characters):

struct nodechar data;struct node *left;struct node *right;;/* This code assumes nodes are identified by single characters

Args: in = inorder listing, post = postorder listing, len = length of listing in charactersReturns: pointer to built tree (of type pointer to node)

*/struct node *buildTree(char *in, char *post, int len)

int i, lenRight, lenLeft; // lenRight (Left) is the length of the inorder listing ofstruct node *p;

if (len < 1) return NULL; // base case: no subtreep=(struct node *)malloc(sizeof(struct node)); // create new nodeif (p == NULL) error("No memory");p->data = post[len-1];p->left = NULL;p->right = NULL;

if (len == 1) return p; // base case - single node tree// find position of last postorder node in inorder listing, i.e current rootfor (i = 0; in[i] != post[len - 1]; i++);lenRight = len - i - 1;lenLeft = len - lenRight - 1;p->left = buildTree(in , post, lenLeft);p->right = buildTree(in + lenLeft +1, post + lenLeft, lenRight);return p;

A similar routine can be developed for pre– and inorder.

/* This code assumes nodes are identified by single charactersArgs: in = inorder listing, pre = postorder listing, len = length of listing in charactersReturns: pointer to built tree (of type pointer to node)

*/struct node *buildTree(char *in, char *post, int len)

int i, lenRight, lenLeft; // lenRight (Left) is the length of the inorder listing ofstruct node *p;

if (len < 1) return NULL; // base case: no subtreep=(struct node *)malloc(sizeof(struct node)); // create new nodeif (p == NULL) error("No memory");p->data = post(len-1);p->left = NULL;

DSAL.105.AC.01.04-B2 5

T.U.C

luj–M

. Jold

os


p->right = NULL;

if (len == 1) return p; // base case - single node tree// find position of last postorder node in inorder listing, i.e current rootfor (i = 0; in[i] != pre[len - 1]; i++);lenRight = len - i - 1;lenLeft = len - lenRight - 1;p->left = buildTree(in , pre + 1, lenLeft);p->right = buildTree(in + lenLeft +1, pre + lenLeft + 1, lenRight);return p;

Ex. 2.4. Consider a hash table that usesten buckets, each containing an unsorted LIFO list of items. The hash functionused adds each character of an input string and then maps the sum to a bucket. Inserted entries are pushed on thefront of the bucket list. Suppose the hash table has an average string length offour characters and an averagebucketlist length of twelve, (i.e., each bucket has approximately twelve entries).

a) Assume the character codes are as follows (A=1, B=2, C=3, D=4, E=5, F=6, G=7, H=8, I=9, J=10, K=11,L=12, M=13, N=14, O=15, P=16, Q=17, R=18, S=19, T=20, U=21, V=22, W=23, X=24, Y=25, Z=26).Compute the selected bucket for the following strings:"C" "JAVA" "FORTRAN"

Solution. "C" 3 "JAVA" 4 "FORTRAN" 2

For the following questions assume that computing the hash function takes an average of four operations andcomparing two strings takes an average of eight operations.Ignore effects of spatial and temporal reference locality.

b) How many of these operations would be required for the average lookup in the hash table described above?(show work)Solution. 4 ops (hash) + (12+1)/2 * 8 ops (search) = 4 + 52 = 56 Operations

c) If instead of a hash table, an unsorted list were used to store the same entries, how many operations would berequired for the average lookup using sequential search? (show work)Solution. (120+1)/2 * 8 = 484 Operations

d) If the hash table were resized to 20 buckets (storing the same entries), how many of these operations would berequired for the average lookup in the hash table described above? (show work)Solution. 4 ops (hash) + (6+1)/2 ops (search) = 4 + 3.5 * 8 = 32 Operations

Ex. 2.5. Collisions and theBirthday Paradox: How many people have to enter a room before there is a chance of more than50% of two persons being born on the same day of the year? (For simplicity, let us ignore the29th of February andlet us assume that births are evenly distributed over the other 365 days of the year.)

Solution. We could hash on the person’s birthday assigning 1 to January1st , 2 to January2nd, and so on, using ahash tableawith 365 locations. Surprisingly, the answer is merely twenty-three! We can derive this number readilyby finding the probability of the opposite outcome, namely thatN people all havedifferentbirthdays:

• For one person this probability obviously is 1.• For two persons, the second person’s birthday can fall on anyof the 365 days except the birth date of the first

person, so the probability is364365 ..• A third person’s birthday can fall on any of the remaining 363days so the probability is364365 ×

363365

Continuing this, we get the general formula fro the probability that allN people have different birthdays:

365× 364× 363 . . . (366−N)

365N

The surprising result is that this value drops below 0.5 forN greater than 22. The lesson is that the search fora perfect hash function is usually pointless: we must be prepared for collisions and so we must modify the basichashing algorithms so that they can handle collisions.

DSAL.105.AC.01.04-B2 6

T.U.C

luj–M

. Jold

os

3 DSA seminar – April 7: Proving Programs Correct


Just as the analysis of algorithms† provides a mathematical foundation for describing the efficiency of programs, there isalso a mathematical foundation for describing the correctness of programs. We will briefly overview some techniques forproving the correctness of programs.

3.1 Types of Errors

There are several types of errors a program may exhibit:

Syntax error:an error in the structure of the program, where it fails to conform to the rules established by the program-ming language.For example, "while )(" is a syntax error in C/C++.

Semantic error:an error in the meaning of the program as expressed in the language. For example, it might be a semanticerror to assign a char constant to a real variable. Some semantic errors (such as most type mismatch errors) can becaught at compile-time; some semantic errors (such as an array index that exceeds the legal bounds) may only becaught at run-time.

Logical error:an error in the algorithm as it is expressed in the program, so that the program fails to perform as desired.For example, this program is incorrect due to a logical error:

/* A program to compute x**y */res = x;for (int i = 1; i < y; i++) res = res * x;

Wheny=0, the program does not perform to specifications (i.e., compute x**y), since it returnsx, and not thecorrect value 1.Syntax errors and semantic errors are intimately related tothe particular programming language involved and canbe detected by the compiler and run-time system.Logical errors are independent of the programming languageand relate only to the problem to be solved. Thecompiler and run-time systemcan notdiscover such errors.

3.2 Testing vs Proof of Correctness

Most programmers usetestingto establish that their program performs to specifications.Testing shows the effect of running the program on a particular set of inputs. As a result, testing is only as good as the testinput. Debugging as we know it is essentially testing. Unfortunately (according to a famous quote by E. Dijkstra) testingcan only show the presence of bugs, not their absence.

A proof of program correctness establishes the correctnessof the program onany input. However, like mathematicalproofs, a proof of program correctness is only as good as the person writing the proof.

3.3 What Can We Prove?

We usually want to prove thata program meets its specifications. That is, if on inputx programP producesy accordingto specifications, thenP is correct. If we have a formal statement of the specifications, we may be able to prove (formally)that the program meets the specifications. The problem with this statement of our goal is that specifications are often:

• informal• incomplete• ambiguous• wrong

To address these problems, there has been much research in formal specifications. For now, we will ignore the problemwith specifications, andfocus on proving that a program meets its specifications.

There are three subgoals in our proof:

Partial correctness: does the program produce the right answers when it halts?Termination: does the program eventually halt?Feasibility: does the program use a reasonable amount of time/space?

†based on http://www.cs.rochester.edu/u/leblanc/csc173.94/correctness.html

DSAL.105.AC.01.04-B2 7

T.U.C

luj–M

. Jold

os


In order to avoid a trial-and-error approach in developing aprogram, we will use the following approach: rather thanattempt to prove anything about a program as a whole, weconsider each small part(e.g., statement) of the program inturn, to see what facts we can establish about each part.

We can attachassertions, that is facts, we believe to be true, to various points of theprogram.

Whenever we reach an assertion, it should be true. Note that most debugging involves attaching invisible assertions tostatements and then using print statements to determine if the assertion is true.

There are several types of assertions:

Postcondition: a predicateP that must hold after execution of a statements is called a postcondition fors, writtens P.Postconditions describe what we knowafter each statement; the last postcondition describes what we know aboutthe output of the program.

Precondition:Q is a precondition for statements with postconditionP , writtenQ s P, iff Q ⇒ P after executionof s. We use preconditions to describe the inputs and initial state of the program.

Invariant: an assertion that is always true. We usually use an assertion within a loop to state what it is true for everyiteration of the loop.

Convergent: a quantity that keeps decreasing as program execution progresses, but which cannot decrease below someknown value. The existence of a convergent is used to provetermination.

3.4 Weakest Precondition

Consider the following program segment, annotated with preconditionQ and postconditionsP :

/* Precondition Q:x >= 0 */x = x + 1;/* Postcondition P:x > 0 */

From this example it is clear that there are many possible preconditions for a given statement and postcondition. Inthis example,x = 12 is a valid precondition, since we can still assert thatx > 0 (the postcondition) after incrementingxwhenx = 12 (the precondition). However,x = 12 is a much more restrictive assumption than what is required to assertx > 0 after incrementingx.

We would like preconditions that are both

• necessary (ie., must be true) and• sufficient (it is all that needs to be true).

We call such a precondition theweakest precondition(wp). In our example,x ≥ 0 is thewp.Definition W is a weakest precondition for statements with postconditionP , writtenwp(s, P ), iff for all other precon-ditionsQ for statements andP , Q ⇒ W . In other words,wp(s, P ) is the weakest precondition for the initial state of acomputation such that execution of statements will result in a properly terminating happening, such that the final state ofthe computation will satisfy postconditionP .

3.5 Example Proof of Correctness

Consider the program below, intended to compute2x, for all x ≥ 0:

int exp(int x) /* Compute 2^x, for nonnegative x *//* Weakest precondition: x >= 0 */res = 1;while (x > 0)

res = res + res;/* Nth iteration invariant: res = 2^N */x = x - 1;/* Convergent: x cannot be less than 0 */

exp = res;/* Postcondition: exp = 2^x */

DSAL.105.AC.01.04-B2 8

T.U.C

luj–M

. Jold

os


Proof of correctness. We can establish the proof of the invariant as follows:

• The first time we reach the invariant,res = 2, which is 21. Assume that on theN th iteration,res = 2N andconsider theN th + 1 iteration, whenres = 2N + 2N = 2N+1. Thus, the invariant holds.• We can establish termination by noting that the convergent,x, must be nonnegative, is ever decreasing, and cannot

be less than 0.• We can establish correctness by noting that, upon termination, either

1. the loop was never executed becausex = 0, soexp = 1 (which is20), or2. the loop was executedN times, soexp = 2x.

3.6 Correctness Proof with Induction

Consider the following program to compute Tower’s of Hanoi,the sequence of moves necessary to moveN > 0 ringsstacked largest to smallest on towerP1 to towerP2 using towerP3 for temporary storage, without putting a big ring ontop of a small one:

void Move (int N, int P1, int P2, int P3)if (N = 1)

printf("Move top of %d to %d\n", P1, P2);else

Move(N-1, P1, P3, P2);printf("Move top of %d to %d\n", P1, P2);Move(N-1, P3, P2, P1);

Move(6,1,2,3)

Note that forMove Pi are integers identifying the three pegs.

To prove termination, notice thatN is a convergent. It cannot be less than 1, since it begins greater than 0, is onlydecremented by 1 on each recursive call, and whenN = 1 recursion stops. Therefore, the depth of recursion is boundedand the algorithm terminates.

To prove partial correctness, we show (via induction) that each call toMove lists a sequence of instructions that movesthe topN rings from towerP1 to P2, possibly usingP3 as storage along the way, leaving all other rings untouched.

Basis step:N = 1. If N = 1, we simply move a ring fromP1 to P2. No other rings are touched.Inductive step: Assume thatMove works correctly onN − 1 rings and show that it works correctly forN rings,N ≥ 2.

Such a call would cause a recursive callMove(N − 1, P1, P3, P2), which by our inductive hypothesis, correctlymoves the topN − 1 rings fromP1 to P3. SinceN ≥ 2, this leaves at least one ring onP1, which is then moved toP2.The final recursive call, by our induction hypothesis, movesN − 1 rings fromP3 to P2, on top of the ring that wasmoved in the previous step. It is also clear that only the topN rings ofP1 have moved, thus completing the proof.Q.E.D.

3.7 Formal Proof of Correctness Using the Invariant Asserti on (Floyd’s) Method

We will consider Binary Search as an example, i.e. Find the index position of elementT in sorted arrayA of lengthNusing binary search. The key idea behind binary search is this: if T is anywhere in arrayA, it must be within a certainrange.

initialize range to be 1..N;loop

Invariant: MustBe(range)if range is empty then

T is not in range; halt;else

compute Mid, the middle of the range;use Mid as a probe to shrink the range;if T is found during shrinking process,return its position;

end loop;

DSAL.105.AC.01.04-B2 9

T.U.C

luj–M

. Jold

os


Based on this algorithm, we can develop a program, ensuring that all actions respect the invariant.

1. /* If T is there, MustBe(1,N) */2. Low = 1; High = N;3. /* MustBe(Low,High) */4. do5. /* MustBe(Low,High) */6. if (Low > High)7. /* Low>High & MustBe(Low,High) => not there */8. position = 0; break;

10. /* MustBe(Low,High) & Low <= High */11. Mid = (Low+High)/2;12. /* MustBe(Low,High) & Low<=Mid<=High */13. if (A[Mid] < T)14. /* MustBe(Low,High) and CantBe(1,Mid) */15. /* MustBe(Mid+1,High) */16. Low = Mid + 1;17. /* MustBe(Low,High) */18. else if (A[Mid] = T)19. /* A[Mid] = T */20. position = Mid; break; 21. else22. /* A[Mid] > T */23. /* MustBe(Low,High) & CantBe(Mid,High) */24. /* MustBe(Low,Mid-1) */25. High = Mid - 1;26. /* MustBe(Low,High) */27. /* MustBe(Low, High) */28. while (true);

Proof of Correctness of Binary Search

First we provepartial correctness. That is, if the program terminates, it does so correctly.

To see this, consider each assertion in turn. In lines 1-3, weinitializeLow andHigh so that the conditionMustBe(Low, High)is true. (MustBe(1, N) is true by definition andMustBe(Low, High) is true ifLow = 1 andHigh = N .) MustBe(Low, High)is our loop invariant, and we have shown it holds when we first enter the loop.

A successful test at line 6 yields the assertion at line 7, so we terminate the loop after settingposition = 0, i.e. T isnot in the array.

If the line 6 test fails, we arrive at line 10, with the invariant unchanged, but knowingLow ≤ High. Line 11 findsthe average ofLow andHigh, truncating to the nearest integer. Since the average is betweenLow andHigh, and sincetruncating can’t move the average belowLow, the invariant at line 12 is established.

Now consider lines 22-26 (lines 13-17 are similar). By arriving at line 22 we knowA[Mid] not equalT andA[Mid] isnot less thanT , soA[Mid] > T , which is the assertion at line 22. The first assertion on line23 must be true since we havedone nothing to change it since line 12, and the second must betrue sinceT < A[Mid] ≤ A[Mid + 1]... ≤ A[High].As a result, the assertion on line 24 must be true, and together with line 25, reestablish the invariant on line 26.

If A[Mid] = T , we exit, correctly settingposition to Mid. Since this is the only other way to exit, terminationcorrectness has been established. (If it terminates, it does so correctly.)

To prove termination, consider the rangeLow..High. The range is initially of finite size,1..N , and lines 6-8 ensure theloop terminates when the range has less than one element. By line 12, we know thatMid is always in the range and lines16 and 25 both removeMid from the range before looping (line 20 causes the loop to terminate). So, the range shrinksby at least one element each iteration. Thus, the rangeLow..High is the convergent and the loop terminates.

3.8 Exercises

Ex. 3.1. Horner’s Rule is an efficient algorithm for evaluating a polynomial at a point. The polynomial is represented byan ordered list,A, of coefficients.x is the point at which we determine the value of the polynomial. The algorithmis:

DSAL.105.AC.01.04-B2 10

T.U.C

luj–M

. Jold

os


Horner(A, x)

1 y ← 02 for i← n downto 03 do y ← A[i] + x · y4 return y

Solution. To show correctness, we want to show that

P (x) = y =

n∑

k=0

akxk (1)

Since the algorithm contains a loop, we will first try to find a useful loop invariant. That is we choose

y =

n−i+1∑

k=0

ak+i+1xk

The idea is that the loop invariant captures the state of the computation at the start of each loop iteration (of lines2–3). We break the proof into three parts: initialization, maintenance, and termination.

Initialization: We show the loop invariant holds before thefirst execution of the loop. Before the first execution ofthe loop,i = n, so the sum runs fromk = 0 to k = n − (n + 1) = −1. Hence the sum is 0. Sincey = 0before the first loop execution, the loop invariant is true.

Maintenance: Assuming the loop invariant is true at the start of execution of thejth loop iteration, we want toshow it is true at the start of thej + 1st loop iteration.Let yj andij be the values ofy andi at the start of thejth loop iteration and letyj+1 andij+1 be the valuesof y andi at the start of thej + 1st loop iteration.At the start of execution of thejth loop iteration,

yj =

n−ij+1∑

k=0

ak+ij+1xk

We are assuming that the loop test is true and the loop body executes, soij ≥ 0.

yj+1 = aij+ x · yj

= aij+ x ·

n−ij+1∑

k=0

ak+ij+1xk

= aij+

n−ij+1∑

k=0

ak+ij+1xk+1

= aij· x0 + aij+1 · x

1 + aij+2 · x2 + . . . + aij+n · x

n−ij (2)

=

n−ij∑

k=0

ak+ijxk

(3)

For line 2, note that whenk = n − (ij + 1), k + ij + 1 = n − (ij + 1) + (ij + 1) = n andk + 1 =n− (ij + 1) + 1 = n− ij .Finally, noting thatij = ij+1 + 1, we have

yj+1 =

n−ij+1+1∑

k=0

ak+ij+1+1xk

which is the loop invariant at the start of thej + 1st loop iteration.Termination: We show that the algorithm is correct (i.e., that equation 1 holds) at the end of the algorithm (using

the loop invariant).Since the loop invariant was true just before execution of the last loop test, and since no variables of the loop

DSAL.105.AC.01.04-B2 11

T.U.C

luj–M

. Jold

os


invariant changed during the loop test, the loop invariant is true. At termination,i = −1. The sum runs fromk = 0 to k = n− (−1 + 1) = n; and the general term isakxk. Hence the loop invariant reduces to

n∑

k=0

akxk

completing the proof.

Ex. 3.2. Verify thepartial correctness of the following algorithm:

/* pre : n>0post : returns the index of the maximum element in A[0...n-1] */int MaxArray(int a[], int n)

1. MaxIndex = 0;2. i = 1;3. while (i != n)

4. if ( a[i] > a[MaxIndex] )5. MaxIndex = i;6. i++;

7. return MaxIndex;

Solution. The loop invariant is "MaxIndex = index of the maximum element inA[0..i− 1]".

Initialization, at lines 1–2, establishes invariance: Initially i = 1 andMaxIndex = 0The loop invariant isMaxIndex = index of the maximum element inA[0..i−1]", that isMaxIndex = indexof the maximum element inA[0..0] = index of the maximum element inA[0] = 0

Maintenance. We show that the loop invariant is preserved bythe iteration, i.e. if is true at the start of the iterationit will be true at the end of the iteration. The code portion atlines 4–6 is the bode of the iteration. Thus wehave:

// MaxIndex == index of maximum element in A[0..i-1]4. if ( a[i] > a[MaxIndex] )5. MaxIndex = i;// MaxIndex == index of maximum element in A[0..i]6. i++;// MaxIndex == index of maximum element in A[0..i-1]

Termination: Loop terminates wheni = n. Together loop invariant and loop termination condition imply thepost condition. As loop terminates wheni == n, the loop invariant, "MaxIndex = index of the maximumelement inA[0..i − 1]" = index of the maximum element in A[0..n-1] (by substituting i = n) which is thepostcondition.

Ex. 3.3. Prove that the following algorithm‡for the addition of natural numbers is correct:

ADD(y, z)

Returny + z wherey, z ∈ N

1 x← 02 c← 03 d← 14 while (y > 0) ∨ (z > 0) ∨ (c > 0)5 do a← y mod 26 b← z mod 27 if a⊕ b⊕ c8 then x← x + d9 c← (a ∧ b) ∨ (b ∧ c) ∨ (a ∧ c)

10 d← 2d11 y ← ⌊y/2⌋12 z ← ⌊z/2⌋13 return x

‡from Ian Parberry and William Gasarch,Problems on Algorithms, second edition, July 2002

DSAL.105.AC.01.04-B2 12

T.U.C

luj–M

. Jold

os


Solution. This correctness proof will make use of the fact that for alln ∈ N,

2⌊n/2⌋+ (n mod 2) = n. (4)

We claim that ify, z ∈ (N), then add(y, z) returns the valuey+z. That is, when line 13 is executed,x = y+z. Foreach of the identifiers, use a subscripti to denote the value of the identifier after theith iteration of the while-loopof lines 4–8, fori ≥ 0, with i = 0 meaning the time immediately before the while loop is entered and immediatelyafter the statement on line 3 is executed. By inspection,

aj+1 = yj mod 2bj+1 = zj mod 2yj+1 = ⌊yj/2⌋zj+1 = ⌊zj/2⌋dj+1 = 2dj

From line 9 of the algorithm,cj+1 is 1 iff§ at least two ofaj+1, bj+1, andcj are 1 (draw table for three variableXOR to see this). Therefore,

cj+1 = ⌊(aj=1 + bj+1 + cj)/2⌋

Note thatd is added intox in line 8 only when an odd number ofa, b, andc are 1. Therefore,

xj+1 = xj + dj((aj+1 + bj+1 + cj) mod 2).

We will now prove the followingloop invariant: For all natural numbersj ≥ 0,

(yj + zj + cj)dj + xj = y0 + z0.

The proof is by induction onj. The basej = 0 is trivial sincec0 = 0, d0 = 1, andx0 = 0. Now suppose that

(yj + zj + cj)dj + xj = y0 + z0.

We are required to prove that

(yj+1 + zj+1 + cj+1)dj+1 + xj+1 = y0 + z0.

By the preceding,

(yj+1 + zj+1 + cj+1)dj+1 + xj+1

= (⌊yj/2⌋+ ⌊zj/2⌋+ ⌊(yj mod 2 + zj mod 2 + cj)/2⌋)2dj

+xj + dj((yj mod 2 + zj mod 2 + cj) mod 2)by Equation 4

= (⌊yj/2⌋+ ⌊zj/2⌋)2dj + xj + dj(yj mod 2 + zj mod 2 + cj)= (yj + zj + cj)dj + xj by Equation 4 twice

Therefore, by the induction hypothesis,

(yj+1 + zj+1 + cj+1)dj+1 + xj+1 = y0 + z0.

This ends the proof of the loop invariant.It remains to prove that the algorithm is correct. We need to prove that the algorithm terminates withx containingthe sum ofy andz. First we will prove that it terminates. By inspection, the values ofy andz are halved (roundingdown if they are odd) on every iteration of the loop. Therefore, they will eventually both have value zero and staythat way. At the first point at whichy = z = 0, eitherc will equal zero orc will be assigned zero on the nextiteration of the loop. Therefore, eventuallyy = z = c = 0 at which point the loop terminates. Now we will provethatx has the correct value on termination. Suppose the loop terminates aftert iterations, for somet ≥ 0. By theloop invariant,

(yj+1 + zj+1 + cj+1)dj+1 + xj+1 = y0 + z0.

Sinceyt = zt = ct = 0, we see thatxt = y0 + z0. Therefore the algorithm terminates withx containing the sumof y andz, as required.

§iff is short for "if and only if"

DSAL.105.AC.01.04-B2 13

T.U.C

luj–M

. Jold

os

4 DSA seminar – April 21: Math and Algorithm analysis


4.1 Strassen Matrix Multiplication

The usual number of scalar operations (i.e., the total number of additions and multiplications) required to performn× nmatrix multiplication is

M(n) = 2n3 − n2 (5)

(i.e. n3 multiplications andn3 − n2 additions). However, Strassen (1969) discovered how to multiply two matrices in

S(n) = 7 · 7lg n − 6 · 4lg n (6)

scalar operations, wherelg is the logarithm to base 2, which is less thanM(n) for n > 45. Forn a power of two (n = 2k),the two parts of (6) can be written

7 · 7lg n = 7 · 7lg 2k

= 7 · 7k = 7 · 2k lg 7 = 7nlg 7 (7)

6 · 4lg n = 6 · 4lg 2k

= 6 · 4k = 6 · (2k)2 = 6n2 (8)

(9)

so (6) becomesS(n) = 7nlg 7 − 6n2 (10)

Two 2× 2 matrices can therefore be multiplied (to yield matrixC = A ·B)[

c11 c12

c21 c22

]

=

[a11 a12

a21 a22

] [b11 b12

b21 b22

]

with only (note that2lg 7 = 7)S(2) = 7 · 2lg 7 − 6 · 22 = 49− 25 = 25 (11)

scalar operations (as it turns out, seven of them are multiplications and 18 are additions). Define the seven products(involving a total of 10 additions) as

Q1 ≡ (a11 + a22)(b11 + b22) (12)

Q2 ≡ (a21 + a22)b11 (13)

Q3 ≡ a11(b12 − b22) (14)

Q4 ≡ a22(−b11 + b21) (15)

Q5 ≡ (a11 + a12)b22 (16)

Q6 ≡ (−a11 + a21)(b11 + b12) (17)

Q7 ≡ (a12 − a22)(b21 + b22) (18)

(19)

Then the matrix product is given using the remaining eight additions as

c11 = Q1 + Q4 −Q5 + Q7 (20)

c21 = Q2 + Q4 (21)

c12 = Q3 + Q5 (22)

c22 = Q1 + Q3 −Q2 + Q6 (23)

(24)

The recurrence relation which arises from Strassen’s matrix multiplication (as multiplications are the costly operations):

T (n) = 7T (n/2) + cn2.

The recursion tree has7k nodes at thek-th level. We get for the successive levels:

cn2

+7c(n

2

)2

+72c(n

4

)2

+73c(n

8

)2

. . .

+7kc( n

2k

)2

DSAL.105.AC.01.04-B2 14

T.U.C

luj–M

. Jold

os


There are aboutlg n levels of this tree (herelg ≡ log2). To grasp the idea, we consider the special case whenn is an exactpower of 2, so that there are exactlylg n levels. Adding up the terms corresponding to recursive calls, we get

cn2

[

1 +7

4+ . . . +

(7

4

)k

+ . . . +

(7

4

)lg n]

At the leaf nodes (bottom of the tree) we have7lg n leaf nodes each contributing some constantb, so that isb · 7lg n =b · 2lg n lg 7 = b · nlg 7. Now consider the summation formula which says that ifα 6= 1 then

m∑

i=0

αi =αm+1 − 1

α− 1=

1− αm+1

1− α(25)

For convenience we write this result forα = 7/4 andm = lg n. We get

T (n) = cn2 αlg n+1 − 1

α− 1+ bnlg 7

≤ cn2αlg n+1 1

α− 1

since making the enumerator larger by dropping the−1 only makes the right hand side larger. We can write 7 as2lg 7 andthenα = 7/4 = 2lg 7−2, soαlg n+1 = 2(lg n+1)(lg 7−2) = 2lg 7−2 · 2lg n(lg 7−2) = 2lg 7−2nlg 7−2. Therefore

T (n) ≤ cn22lg 7−2nlg 7−2 1

α− 1+ bnlg 7

= 2lg 7−2cnlg 7 1

α− 1+ bnlg 7

= Θ(nlg 7)

There is nothing special about the numbers 7 and 2 in7T (n/2) in this calculation. The 2 means that the answer willinvolve lg, i.e the logarithm to the base 2 (log2). If 2 is replaced by something else we get the logarithm to that base. Thepower ofn will be the logarithm to that base of whatever number occurs in the place where 7 is here.For the calculation it was important that the exponent 2 in the cn2 term of the recursion is less thanlg 7. Say it wasMinstead of 2. Then what would happen is that we would get from the recursive levels the following:

cnM

[

1 +7

2M+ . . . +

(7

2M

)k

+ . . . +

(7

2M

)lg n]

≤ cnM 1− αlg n+1

1− α

where nowα = 7/2M . But if M > lg 7 thenα < 1. So numerator and denominator are negative. We have to writetheright-hand side as

cnM 1− αlg n+1

1− α

to make the numerator and denominator positive. Then to makethe denominator larger we drop the negative term and get

T (n) = O(nM )

If M = lg 7 we getT (n) = Θ(nlg 7 lg n). Compare this result to what the Master Theorem says.

4.2 Exercises

Ex. 4.1. Solve the following recurrences, without using theMaster Formula, and give your answer inΘ notation. For eachof them, assume that the base case isT (k) = 1, k ≤ 2.

a) T (n) = 3T (n/7) + 6n2

b) T (n) = 20T (n/2) + n3

c) T (n) = 3T (n− 1) + 2004

DSAL.105.AC.01.04-B2 15

T.U.C

luj–M

. Jold

os


d) T (n) = T (n− 1) + 1/n

Also, specify the ones for which the Master Formula could be used.Solution.

a) One way to solve is byunrolling or expansion. The unrolling or expansion method of solving a recurrenceinvolves repeating plugging in the recurrence for lower values until a pattern emerges. It is usually a goodidea to confirm the pattern using mathematical induction. For our recurrence we get:

T (n) = 3T (n/7) + 6n2

= 3(3T (n/7) + 6(n/7)2) + 6n2

= 3(3(3T (n/72) + 6(n/72)2) + 6(n/7)2) + 6n2

= . . .

(26)

Now, think of the recursion tree. The recursion tree has3k nodes on thekth level. We get for the successivelevels:

6n2

+3 · 6(n/7)2

+32 · 6(n/72)2

+33 · 6(n/73)2

. . .

+3k · 6(n/7k)2

For this recurrence we have:• Subproblem size at leveli is n

7i

• Subproblem size hits 1 when1 = n7i ⇒ i = log7 n

• Cost of a node at leveli = 3i ⇒ last level has3log7 n = nlog7 3 nodes• The total cost will then be:

T (n) =

log7 n−1∑

i=0

(3

49

)i

6n2 + Θ(nlog7 3)

≤

∞∑

i=0

(3

49

)i

6n2 + Θ(nlog7 3)

=1

1− 349

6n2 + Θ(nlog7 3 = O(n2)⇒ T (n) = O(n2)

If allowed we could also use the Master Formula here, witha = 4, b = 7, f(n) = 6n2. This falls into case3 of Master Theorem (Reminder: forT (n) = aT (n/b) + f(n), wherea ≥ 1, b ≥ 1: if ∃ε > 0 such thatf(n) = Ω(logb a+ε) thenT (n) = Θ(nlogb a)).

b) Building the recursion tree, and adding up level by level,we get

T (n) = n3[1 + 20/23 + (20/23)2 + (20/23)3 + . . . + (20/23)log2 n

]

and taking(20/23)log2 n as a common factor we get

T (n) = n3(20/23)log2 n[(23/20)log2 n + . . . + (23/20) + 1

]

which impliesT (n) = Θ(n3(20/23)log2 n). Using the fact thataloga x = x anday loga x = xy (see that1/23 log2 n = 1/n3; also becauselogb x/ logb a = loga x we have thatalogb x = xlogb a) and we getT (n) =Θ(20log2 n) = Θ(nlog2 20). Again, here we could (if allowed) use the Master Formula.

c) By unrolling the recurrence we get

T (2) = 3T (1) + 2004 = 3 + 2004 sinceT (1) = 1

T (3) = 3T (2) + 2004

T (4) = 3T (3) + 2004 = 3(3T (2) + 2004) + 2004

T (5) = 3T (4) + 2004 = 3(3T (3) + 2004) + 2004 = 3(3(3T (2) + 2004) + 2004) + 2004

. . .

DSAL.105.AC.01.04-B2 16

T.U.C

luj–M

. Jold

os


Thus, we get:T (n) = 2004

[30 + 31 + . . . + 3n−3

]+ 3n−2T (2)

Therefore (using (25))

T (n) = 20043n−2 − 1

2+ 3n−2

which implies thatT (n) = Θ(3n)

d) By unrolling the recurrence we get

T (n) =1

n+

1

n− 1+ . . . +

1

3+ T (2) = Hn −

1

2

As an approximation forHn for finite n is Hn = lnn + 0.5772156649¶ we can state that

T (n) = Θ(log n)

Ex. 4.2. Consider the algorithm presented in class:

FASTPOWER(a, n)

Input: integersa, n Output: returnsan

if n = 0then return 1;

if n mod 2 = 0then p = FASTPOWER(a, n/2)

return p · pelse p = FASTPOWER(a, (n− 1)/2)

return a · p · p

Let T (k) be the number of primitive operations required to com-pute an wheren = 2k. Give a recurrence equation that describesT (k) in terms ofT (k− 1). Guess the explicit formula forT (k).Prove your guess by induction onk.

Solution. We haveT (0) = 10 because whenn = 20 = 1, the algorithm calls itself recursively once, and if youcount all primitive operations, you get 10. Fork > 0, we getT (k) = 9 + T (k − 1). Doing the substitution trickyields the guessT (k) = 10 + 9k. We can then prove this by induction onk.

Base case: Whenk = 0, the definition ofT (n) says thatT (0) = 10. This is equal to10 + 9 · 0.Induction Hypothesis: Assume thatT (b) = 10 + 9b. We need to prove that this implies that

T (b + 1) = 10 + 9(b + 1).

We have:

T (b + 1) = 9 + T (b) by definition ofT

= 9 + 10 + 9b by the induction hypothesis

= 10 + 9(b + 1)

Thus, we have proved thatT (k) = 10 + 9k for all k ≥ 0. Now, since we chosen = 2k, we havek = log n, so thetime to compute an is10 + 9 log n which isO(log n).

Ex. 4.3. Prove thatlog(n!) ∈ Θ(n log n)Solution. First, we show thatlog(n!) ∈ O(n log n). For this, we need to findc andn0 such that for alln ≥ n0,log(n!) ≤ cn log n. This is the easy part, asc = 1 andn0 = 1 will work:

log(n!) = log(n× (n− 1)× (n− 2)...× 2× 1)

= log(n) + log(n− 1) + log(n− 2) + ... + log(2) + log(1)

≤ log(n) + log(n) + log(n) + ... + log(n) + log(n)

= n log(n)

= cn log(n)

(27)

Thus,log(n!) ∈ O(n log n).Now, to prove thatn log n ∈ O(log(n!)), we need to findc andn0 such that for alln ≥ n0, n log n ∈ O(log(n!)).

¶0.5772156649 is known as Euler’s constant (or the Euler-Mascheroni constant)

DSAL.105.AC.01.04-B2 17

T.U.C

luj–M

. Jold

os


This time, it will be easier to start with the right side of theinequality. Let’s not pickc andn0 yet and see what wewould need.

log(n!) = log(n) + log(n− 1) + log(n− 2) + ... + log(n/2 + 1)︸︷︷︸

≥n/2 log(n/2)

+ log(n/2) + ... + log(2) + log(1)︸︷︷︸

n/2 log 1

= n/2 log(n/2)

= n/2(log n− 1)

≥ n/2(log n−1

2log n) if n ≥ 4

= 1/4n logn

Thus, forn0 = 4, c = 4, we get thatn logn ≤ 4n logn for all n ≥ n0.Ex. 4.4. Prove using only the definition ofO() that(n + 10)1.5 + n + 1 is O(n1.5).

Solution. We need to find ac ∈ R and ann0 ∈ N such that ifn ≥ n0, then(n +10)1.5 + n +1 ≤ cn1.5. Let’s pickn0 = 10 and see what constantc we will need. Ifn ≥ 10, we have:

(n + 10)1.5 + n + 1 ≤ (n + n)1.5 + n + 1 sincen ≥ 10 (28)

≤ (2n)1.5 + n1.5 + n1.5 sincen ≥ 10 (29)

= (2 + 21.5)n1.5 (30)

Thus, if we choosec = 2 + 21.5 andn0 = 10, we have that(n + 10)1.5 + n + 1 ≤ cn1.5. Thus(n + 10)1.5 + n + 1is O(n1.5).

Ex. 4.5. Prove, using any valid method you want, that2n is notO(n2).Solution. The simplest thing to do is to take the limit of the ratio of thetwo functions and use limit whenn→ +∞to determine if one is big-Oh of the other:

limn→+∞

2n

n2= lim

n→+∞

d2n/dn

dn2/dnL’Hôpital rule

= limn→+∞

2n ln 2

2n

= limn→+∞

d(2n ln 2)/dn

d(2n)/dnL’Hôpital again

= limn→+∞

2n(ln 2)2

2= +∞

Since the ratio of the two functions diverges to+∞, we can conclude that2n is notO(n2).

DSAL.105.AC.01.04-B2 18

T.U.C

luj–M

. Jold

os

5 DSA seminar – May 5: Algorithm Design


5.1 Using Dynamic Programming to Solve the 0–1 Knapsack Prob lem

As we have already seen in class, the classical (0–1) knapsack problem is a famous optimization problem. A thief isrobbing a store, and findsn items which can be taken. Theith item is worthvi eand weighswi pounds, wherevi andwi

are integers. He/she wants to take as valuable a load as possible, but has a knapsack that can only carryW total pounds.Which items should he take? (The reason that this is called 0-1 knapsack is that each item must be left (0) or taken (1)in it entirety. It is not possible to take part of an item or multiple copies of an item.) This optimization problem arises inindustrial packing applications. For example, you may wantto ship some subset of items on a truck of limited capacity.More formally, givenv1, v2, . . . , vn and w1, w2, . . . , wn, and W > 0, we wish to determine the subsetT ⊆1, 2, . . . , n (of objects to “take”) that maximizes

∑

i∈T

vi,

subject to∑

i∈T

wi ≤W

Let us assume that thevi’s, wi’s andW are all positive integers. We cannot really hope to find an efficient solution. Weassume that thewi’s are small integers, and thatW itself is a small integer. We show that this problem can be solved inO(nW ) time. (Note that this is not very good ifW is a large integer. But if we truncate our numbers to lower precision,this gives a reasonable approximation algorithm.)Here is how we solve the problem. We construct an arrayV [0..n, 0..W ]. For1 ≤ i ≤ n, and0 ≤ j ≤ W , at the entryV [i, j] we will store the maximum value of any subset of objects1, 2, . . . , i that can fit into a knapsack of weightj. Ifwe can compute all the entries of this array, then the array entry V [n, W ] will contain the maximum value of alln objectsthat can fit into the entire knapsack of weightW .To compute the entries of the arrayV we will imply an inductive approach. As a basis, observe thatV [0, j] = 0 for0 ≤ j ≤W since if we have no items then we have no value. We consider twocases:

Leave object i : If we choose to not take objecti, then the optimal value will come about by considering how tofill aknapsack of sizej with the remaining objects1, 2, . . . , i− 1. This is justV [i− 1, j].

Take object i : If we take objecti, then we gain a value ofvi but have used upwi of our capacity. With the remainingj − wi capacity in the knapsack, we can fill it in the best possible way with objects1, 2, . . . , i − 1. This isvi + V [i− 1, j − wi]. This is only possible ifwi ≤ j.Since these are the only two possibilities, we can see that wehave the following rule for constructing the arrayV .The ranges oni andj arei ∈ [0..n] andj ∈ [0..W ].

V [0, j] = 0

V [i, j] =

V [i− 1, j] if wi > jmax(V [i− 1, j], vi + V [i− 1, j − wi]) if wi ≤ j

The first line states that if there are no objects, then there is no value, irrespective ofj. The second line implements therule above.It is very easy to take these rules an produce an algorithm that computes the maximum value for the knapsack in timeproportional to the size of the array, which isO((n + 1)(W + 1)) = O(nW ). The algorithm is given below.

KNAPSACK01(v, w, W )

1 n← length[v]2 for j ← 0 to W3 do V [0, j]← 0 initialization4 for i← 1 to n5 do for j ← 0 to W6 do leaveV al← V [i− 1, j] total value if we leavei7 if j ≥ wi enough capacity to takei8 then takeV al← vi + V [i− 1, j − wi] total value if we takei9 else takeV al← −∞ cannot takei

10 V [i, j]← max(leaveV al, takeV al) final value is maximum11 return V [n, W ]

DSAL.105.AC.01.04-B2 19

T.U.C

luj–M

. Jold

os


Capacity→ j = 0 1 2 3 4 5 6 7 8 9 10Item Value Weight

1 10 5 0 0 0 0 0 10 10 10 10 10 102 40 4 0 0 0 0 40 40 40 40 40 50 503 30 6 0 0 0 0 40 40 40 40 40 50 704 50 3 0 0 0 50 50 50 50 90 90 90 90

Final result isV [4, 10] = 90 (for taking items 2 and 4).

Figure 5: 0–1 Knapsack Example.

An example is shown in Figure 5. The final output isV [n, W ] = V [4, 10] = 90. This reflects the selection of items 2 and4, of valuese40 ande50, respectively and weights4 + 3 ≤ 10.Values of the objects are 10, 40, 30, 50. Weights of the objects are 5, 4, 6, 3.

The only missing detail is what items should we select to achieve the maximum. We will leave this as an exercise. Theykey is to record for each entryV [i, j] in the matrix whether we got this entry by taking theith item or leaving it. With thisinformation, it is possible to reconstruct the optimum knapsack contents.

5.2 Recursion Elimination

5.2.1 Tail Recursion Elimination

If the body of a recursive routineendswith a recursive call to the routine, then we can replace the call with a setting ofparameters followed by a branch to the start of the routine body. This:

• Saves routine call and return overhead.• Saves activation record allocation/deallocation overhead and space.

There are some issues to take care of:

• Elimination needs extreme care to preserve parameter passing semantics.• Elimination is often done where parameters are passed by value.

An example is shown below:

int Func( int A , int B ) int Func( int A , int B )top:

... ...return Func( A - 1 , B + 1 ); A = A - 1 ;

B = B + 1 ;goto top ;

A recursive call istail-recursiveiff the first instruction after returning from it isreturn. A stack is not needed becauseeverything on the stack will be discarded. Tail-recursive functions can be rewritten using iteration. Note that the recursivecall in fact is not tail-recursive:

int fact (int n)

if (n==0) then return 1;else return n * fact(n-1);

The first instruction after returning from the recursive call is a multiplication.

5.2.2 Tail Recursion Example: Binary Search

Consider binarySearch:

int binarySearch ( int T[], int a, int b, key K)

if (a > b) return -1;middle = (a + b)/2 // note is an integer divisionif (K == T[middle]) return middle;elseif (K < T[middle]) return binarySearch(T, a, middle-1, K);

DSAL.105.AC.01.04-B2 20

T.U.C

luj–M

. Jold

os


elseif (K > T[middle]) return binarySearch(T, middle+1, b, K);

Both recursive calls are tail-recursive, and can be eliminated as follows:

int binarySearch ( int T[], int a, int b, key K)start:if (a > b) return -1;middle = (a + b)/2 // note is an integer divisionif (K == T[middle]) return middle;elseif (K < T[middle])

// was return binarySearch(T, a, middle-1, K);b=middle-1;

elseif (K > T[middle])

// was return binarySearch(T, middle+1, b, K);a=middle+1;

goto start;

5.2.3 Elimination of Non-Tail Recursion

We can easily simulate recursion using a stack, thegotostatement, and theif statement.

Example. Let’s eliminate recursion in

int fact (int n)

if (n<=1) return 1;else return n*fact(n-1);

The algorithm for recursion elimination is:

1. Expand all expressions containing recursive calls so that they are placed in temporary variables.

int fact (int n)

int tmp;if (n<=1) return 1;elsetmp = fact(n-1);return n*tmp;

2. Assign all expressions in return statements to a specific temporary variable, e.g.,ret before return.

int fact (int n)

int tmp;int ret;if (n<=1)ret = 1;return ret;

elsetmp = fact(n-1);ret = n*tmp;return ret;

DSAL.105.AC.01.04-B2 21

T.U.C

luj–M

. Jold

os


3. Create astruct of the all variables except/ret, but also include a variablewhere that indicates where to return to. In thiscase, where has only one value (1).

struct Frame

int n;int tmp;int where;

;

4. Create astackof struct Frame, by whatever method; definepushandpopmethods operating onstackthat save and restorevalues in these variables. Also implementemptythat is 1 if the stack is empty (you already know this ADT):

push(int n,int tmp, int where);pop(int &n, int &tmp, int &where);empty();

5. Then:

a) for each recursive call, replace it with apush of all variables, followed by re-assignmentand abranch to the topof theroutine; assume that the return value is inret. Label the return point uniquely. Note that thewhere variable is not needehere as there is a single return point. The code:

tmp = fact(n-1);

becomes

push(n,tmp,1); n=n-1; goto start;back1:

tmp=ret;

b) For each return statement, replace it with code that checks whether the stack is empty, and returns if it is. If it is not,thenput the return value inret and pop the old values of locals.

return ret;

becomes

if (!empty())

pop(&n,&tmp); goto back1;else

return ret;

6. Thus the code for factorial becomes:

int fact (int n)

int tmp;int ret;

start:if (n<=1)

ret = 1;if (!empty())


return ret;

else

push(n,tmp);goto start;

back1:

DSAL.105.AC.01.04-B2 22

T.U.C

luj–M

. Jold

os


tmp=ret;ret=n*tmp;if (!empty())


return ret;

For a more complex example, consider:

int fib(int n)

if (n<=2) return 1;else return fib(n-1)+fib(n-2);

• First, we add enough temporaries to disambiguate the calls,and put return values into a distinguished location.

int fib(int n)int ret,tmp1,tmp2;if (n<=2)

ret=1;return ret;

else

tmp1 = fib(n-1);tmp2 = fib(n-2);ret = tmp1+tmp2;return ret;

• Then we distinguish the calls by adding awherevalue of 1 or 2 to push and pop:

int fib(int n)int ret,tmp1,tmp2;start:if (n<=2)

ret=1;return ret;

else

push(n,tmp1,tmp2,1);n=n-1;goto start;

back1:tmp1 = ret;push(n,tmp1,tmp2,2);n=n-2;goto start;

back2:tmp2=ret;ret = tmp1+tmp2;return ret;

DSAL.105.AC.01.04-B2 23

T.U.C

luj–M

. Jold

os


• Finally, adjust returns so that they go to the appropriate place.

int fib(int n)int ret,tmp1,tmp2,where;start:if (n<=2)

ret=1;if (!empty())

pop(&n,&tmp1,&tmp2,&where);switch (where)

case 1: goto back1; break;case 2: goto back2; break;

else

return ret;

else

push(n,tmp1,tmp2,1);n=n-1;goto start;

back1:tmp1 = ret;push(n,tmp1,tmp2,2);n=n-2;goto start;

back2:tmp2=ret;ret = tmp1+tmp2;if (!empty())

pop(&n,&tmp1,&tmp2,&where);switch (where)

case 1: goto back1; break;case 2: goto back2; break;

else

return ret;

DSAL.105.AC.01.04-B2 24

1 dsa seminar – march 10

Documents