1

Midterm Review

2

Midterm Exam

• Thursday, October 15 in classroom

• 75 minutes

• Exam structure:

– TRUE/FALSE questions

– short questions on the topics discussed in class

– homework-like problems

3

Asymptotic Notations

• A way to describe behavior of functions in the limit

– Abstracts away low-order terms and constant factors

– How we indicate running times of algorithms

– Describe the running time of an algorithm as n grows to

• O notation: asymptotic “less than”: f(n) “≤” g(n)

• notation: asymptotic “greater than”: f(n) “≥”

g(n)

• notation: asymptotic “equality”: f(n) “=” g(n)

4

Exercise

• Order the following 6 functions in increasing order of their growth rates: equivalence class iff f(n) = Θ(g(n)) – n log10n, 4lg n, n2, nn, 3n + n2 .

a=0 f(n) = o(g(n) O(g(n)) but not Θ(g(n))a= f(n) = ω(g(n)) Ω(g(n)) but not Θ(g(n)) 0<a< f(n) = Θ(g(n))

– n log10n, 4lg n, n2, 3n + n2 , nn

)(

)(lim ng

nfa

n

5

Analyzing Algorithms

• Alg.: MIN (a[1], …, a[n]) m ← a[1];

for i ← 2 to n if a[i] < m

then m ← a[i];

• Running time: – the number of primitive operations (steps) executed before

terminationT(n) =1 [first step] + (n) [for loop] + (n-1) [if condition] + (n-1) [the assignment in then] = 3n - 1

• Order (rate) of growth: – The leading term of the formula– Expresses the asymptotic behavior of the algorithm

• T(n) grows like n

6

Running Time Analysis

• Algorithm Loop2(n)p=1for i = 1 to 2n

p = p*i

• Algorithm Loop3(n)p=1for i = 1 to n2

p = p*i

O(n)

O(n2)

7

Running Time Analysis

Algorithm Loop4(n)s=0for i = 1 to 2n

for j = 1 to is = s + i

O(n2)

8

Recurrences

Def.: Recurrence = an equation or inequality that describes a function in terms of its value on smaller inputs, and one or more base cases

• Recurrences arise when an algorithm contains recursive calls to itself

• Methods for solving recurrences– Substitution method– Recursion tree method– Master method

9

Example Recurrences

• T(n) = T(n-1) + n

– Recursive algorithm that loops through the input to eliminate one item

• T(n) = T(n/2) + c

– Recursive algorithm that halves the input in one step

Θ(n2)

Θ(lg n)

10

Analyzing Divide and Conquer Algorithms

• The recurrence is based on the three steps of the paradigm:

– T(n) = running time on a problem of size n– Divide the problem into a subproblems, each of size

n/b: takes– Conquer (solve) the subproblems: takes– Combine the solutions: takes

(1) if n ≤ c

T(n) =

D(n)aT(n/b)

C(n)

aT(n/b) + D(n) + C(n) otherwise

11

Master Theorem

Theorem 4.1

Let a 1 and b > 1 be constants, let f(n) be a function, and Let T(n) be defined on nonnegative integers by the recurrence T(n) = aT(n/b) + f(n), where we can replace n/b by n/b or n/b. T(n) can be bounded asymptotically in three cases:

1. If f(n) = O(nlogba–) for some constant > 0, then T(n) = (nlogba).

2. If f(n) = (nlogba), then T(n) = (nlogbalg n).3. If f(n) = (nlogba+) for some constant > 0,

and if, for some constant c < 1 and all sufficiently large n,

we have a·f(n/b) c f(n), then T(n) = (f(n)).

Theorem 4.1

Let a 1 and b > 1 be constants, let f(n) be a function, and Let T(n) be defined on nonnegative integers by the recurrence T(n) = aT(n/b) + f(n), where we can replace n/b by n/b or n/b. T(n) can be bounded asymptotically in three cases:

1. If f(n) = O(nlogba–) for some constant > 0, then T(n) = (nlogba).

2. If f(n) = (nlogba), then T(n) = (nlogbalg n).3. If f(n) = (nlogba+) for some constant > 0,

and if, for some constant c < 1 and all sufficiently large n,

we have a·f(n/b) c f(n), then T(n) = (f(n)).

Whether the problem cost is dominated by the root or by the leaves

12

ExampleTwo different divide-and-conquer algorithms A and B have been designedfor solving the problem .

– A partitions into 4 subproblems each of size n/2, where n is the input size for , and it takes a total of (n1.5) time for the partition and combine steps.

– B partitions into 4 subproblems each of size n/4, and it takes a total of (n) time for the partition and combine steps.

Which algorithm is preferable? Why?

A: , (case 1)

B: , (case 2)

)(4

4)(2

4 5.1 nn

TTnn

TT BA

5.1)( nnf 24loglog 2 nnn ab )()( 2 nOnf 2)( nnT

nnf )( nnn ab 2loglog 2 )()( nnf nnnT lg)(

13

Exercise

T(n)=3T(n/2)+ n lg n Case 1: T(n) =Θ(nlg 3)

T(n)=4T(n/2)+ n2 √n Case 3: T(n) =Θ(n2 √n)

14

Sorting

• Insertion sort– Design approach:– Sorts in place:– Best case:– Worst case: – n2 comparisons, n2 exchanges

• Bubble Sort– Design approach:– Sorts in place:– Running time:– n2 comparisons, n2 exchanges

Yes(n)

(n2)

incremental

Yes(n2)

incremental

15

Sorting

• Selection sort– Design approach:– Sorts in place:– Running time: – n2 comparisons, n exchanges

• Merge Sort– Design approach:– Sorts in place:– Running time:

Yes

(n2)

incremental

No(nlgn)

divide and conquer

16

Quicksort• Quicksort

– Idea:

– Design approach:

– Sorts in place:

– Best case:

– Worst case:

• Partition– Running time

• Randomized Quicksort

Yes(nlgn)

(n2)

Divide and conquer

(n)

Partition the array A into 2 subarrays A[p..q] and A[q+1..r], such that each element of A[p..q] is smaller than or equal to each element in A[q+1..r]. Then sort the subarrays recursively.

(n lg n) – on average(n2) – in the worst case

17

Heap Data Structure

• Def: A heap is a nearly complete binary tree with the following two properties:– Structural property: all levels are full, except possibly the

last one, which is filled from left to right– Order (heap) property: for any node x, parent(x) ≥ x

Heap

5

7

8

4

2

18

Array Representation of Heaps

• A heap can be stored as an array A.

– Root of tree is A[1]

– Parent of A[i] = A[ i/2 ]

– Left child of A[i] = A[2i]

– Right child of A[i] = A[2i + 1]

– Heapsize[A] ≤ length[A]

• The elements in the subarray

A[(n/2+1) .. n] are leaves

• The root is the max/min element of the

heap

A heap is a binary tree that is filled in order

19

Operations on Heaps

– MAX-HEAPIFY O(lgn)

– BUILD-MAX-HEAP O(n)

– HEAP-SORT O(nlgn)

– MAX-HEAP-INSERT O(lgn)

– HEAP-EXTRACT-MAX O(lgn)

– HEAP-INCREASE-KEY O(lgn)

– HEAP-MAXIMUM O(1)

– You should be able to show how these algorithms perform on

a given heap, and tell their running time

20

Questions

• TRUE FALSE

A reverse sorted array is always a max heap.

• What is the maximum number of nodes possible in a max heap of height

h?

- max number reached when all levels are full

h

i

i

0

2 1212

12 11

h

h

21

Questions

• TRUE FALSE

The sequence 23, 17, 14, 6, 13, 10, 1, 5, 7, 12 is a heap.

7, which is the right child of 6, is greater than its parent.

22

HW3. 1

• Since a heap is an almost-complete binary tree, it has at most 2h+1-1 elements (if it is complete) and at least 2h-1+1=2h elements (if the lowest level has just 1 element and the other levels are complete)

• Any leaf can be the smallest element in a max-heap (tree representation). A[n/2+1,…,n] are leaves in a max-heap (array representation)

• If the array is sorted in non-increasing order, it is a max-heap. If the array is sorted in non-decreasing order, it is a min-heap. If [23 17 41 6 13 10 1 5 7 12] a max heap– No: 7 is a child of 6

23

HW3. 2

• Since we call Max-Heapify on A[i] in the loop, and Max-Heapify requires that the subtrees rooted at both children of A[i] are max-heaps, we must decrease from length[A]/2 to 1. If we increase from 1, the two subtrees rooted at A[2] and A[3]are not max-heaps yet.

24

25

Readings

• Chapter: 1-4, 6-9, 12-13• Appendix: A, C