CS

C 3

31

: A

lgori

thm

An

aly

sis

Divide-and-Conquer Algorithms

The divide-and-conquer strategy solves a problem by:

Breaking it into subproblems that are themselves smaller instances of the same type

of problem.

Recursively solving these subproblems.

Appropriately combining their answers.

CS

C 3

31

: A

lgori

thm

An

aly

sis

Recurrence Relations

Divide-and-conquer algorithms often follow a generic pattern:

They tackle a problem of size n by recursively solving a subproblems of size n/b and then

combining these answers in O(nd) time, for some a, b, d > 0.

Their running time can therefore be captured by the equation T(n) = aT(⎡n/b⎤) + O(nd).

If T(n) = aT(⎡n/b⎤) + O(nd) for some constants a > 0, b > 1, and d ≥ 0, then

The Master Theorem tells us the running times of most of the divide-

and-conquer procedures we are likely to use.

CS

C 3

31

: A

lgori

thm

An

aly

sis

Binary Search

Given a sorted array of numbers, return the index of the number k or -1 if it’s not there.

We could go through each number and check to see if that number is k. If so, return the index.

What’s the running time?

Instead, we want to USE the fact that the array is sorted.

What is a better way? (Think high/low game).

2 3 5 6 7 13 19 100

Input: Look for the 13 in:

13 19 100

13

Look at middle number, 7. 13>7, so recurse on second half:

Look at middle number, 19. 13<19, so recurse on first half:

Look at middle number, 13. l found it, with only 3 recursions.

Search in part of Array A for element k, and return its index. Assumes A is sorted.

function binarySearch(A[left..right], k)

// Base case: 0 elements left, I didn’t find it.if right< left return Not foundmid<- (right-left/2) + leftif k = A[mid] return midif k < A[mid] return A[left..mid-1]if k > A[mid] return A[mid+1..right]

What is the recurrence relation?

What is the running time?

function binarySearch(A[left..right], k)

// Base case: 0 elements left, I didn’t find it.if right< left return Not foundmid<- (right-left)/2 + leftif k = A[mid] return midif k < A[mid] return A[left..mid-1]if k > A[mid] return A[mid+1..right]

CS

C 3

31

: A

lgori

thm

An

aly

sis

Mergesort

The problem of sorting a list of numbers lends itself immediately to a divide-and-conquer

strategy.

Split the list into two halves.

Recursively sort each half.

Merge the two sorted sublists.

10 2 5 3 7 13 1 6

Input:

10 2 5 3 7 13 1 6

10 2 5 3 7 13 1 6

10 2 5 3 7 13 1 6

10 2 5 3 7 13 1 6

1 2 3 5 6 7 10 13

2 3 5 10 1 6 7 13

2 10 3 5 7 13 1 6

function mergesort(a[1...n])

if n > 1:return merge(mergesort(a[1... n/2 ]),⎣ ⎦ mergesort(a[ n/2 + 1...n]))⎣ ⎦

else:return a

function merge(x[1...k], y[1...m])

if k = 0: return y[1...m]if m = 0: return x[1...k]

if x[1] ≤ y[1]:return x[1] merge(x[2...k], y[1...m])✪

else:return y[1] merge(x[1...k], y[2...m])✪

where is concatenation.✪

function merge(x[1...k], y[1...m])

if k = 0: return y[1...m]if m = 0: return x[1...k]

if x[1] ≤ y[1]:return x[1] merge(x[2...k], y[1...m])✪

else:return y[1] merge(x[1...k], y[2...m])✪

This merge procedure does a constant amount of work per recursive call (provided the required

array space is allocated in advance), for a total running time of O(k + m).

Thus merge’s are linear, and the overall time take by mergesort is

T(n) = 2T(n/2) + O(n).

a = 2, b = 2, d = 1 ➠ O(n log n)

CS

C 3

31

: A

lgori

thm

An

aly

sis

Medians

What is the median of [45, 1, 10, 30, 25]?

How do you compute the median of n numbers?

Sorting takes O(n log n), can we do better?

25

Sorting is doing far more work than we really need -- we just want the middle element and don’t care about the relative ordering of the

rest of them.

When looking for a recursive solution, it is paradoxically often easier to work with a more general version of the

problem.

Selection

Input: A list of numbers S; an integer k

Output: The kth smallest element of S

For instance, if k = 1, the minimum of S is sought.

For median, k = ⎣⎜S⎟/ 2⎦.

For any number v, split list S into three categories:

elements smaller than v (SL)

elements equal to v (SV)

elements greater than v (SR)

The three sublists can be computed in linear time O(n).

For instance, if the array

2 36 5 21 8 13 11 20 5 4 1S:

is split on v = 13, the three subarrays generated are:

2 5 8 11 5 4 1SL:

13Sv:

36 21 20SR:

Now we can instantly narrow down the search to one of these sublists.

If we wanted, say, the ninth-smallest element of S, we know it must be the first-

smallest element of SR since | SL | + | SV | = 8.

2 5 8 11 5 4 1SL:

13SV:

36 21 20SR:

selection(S,9) = selection(SR,1)

if k ≤ | SL |selection(S,k) = selection(SL, k)

if | SL | < k ≤ | SL | + | SV |selection(S,k) = v

if k > | SL | + | SV |selection(S,k) = selection(SR, k - | SL | - | SV |)

If we could always pick v so that ⎜SL⎟, ⎟ SR⎟ ≈ 1/2(S)

T(n) = T(n/2) + O(n).

a = 1, b = 2, d = 1 ➠ O(n)

But this requires picking v to be the median, which is our ultimate goal!

We will pick v randomly from S.

n + (n - 1) + (n - 2) + ... + n/2 = Θ(n2)

Naturally, the running time of our algorithm depends on the random choice of v.

The worst-case scenario would be if we kept picking v to be the largest (or

smallest) element of the array.

Θ(n)

The best-case scenario would be if we picked v to be the median element of the

array.

Where does the average running time lie?

v is good if it lies within the 25th to 75th percentile of the array.

A good v ensures that the sublists SL and SR have size at most three-fourths that of

S.

Half of the elements of any list must fall between the 25th and 75th percentile.

Lemma: On average a fair coin needs to be tossed two

times before a “heads” is seen.

After two split operations on average, the array will shrink to at most three-fourths of its size.

Letting T(n) be the expected running time on an array of size n, we get

T(n) ≤ T(3n/4) + O(n).

a = 1, b = 4/3, d = 1 O(n)➠

So, the overall ideas:

Often the obvious way to solve a problem is not the most efficient.

Try to find a way to divide the problem into pieces.

IF you know the answer (recursively) to the piece, does that help you find the overall answer?

Once you write a recursive function, see if its solution is quicker than your iterative way.

Practice:

(Book, problem 2.23)

An array A[1..n] has a majority element if one element occurs in more than half of the entries. Design an O(nlgn) algorithm to decide if the array has a majority element and if so, what the element is. You cannot sort the array, or ask if A[i]>A[j], since comparisons are not supported on the array type. (For example, if the array were of GIF files.) You can, however, ask if A[i] = A[j].