divide and conquer (part ii) multiplication of two numbers let u = (u 2n-1 u 2n-2 … u 1 u 0 ) 2...

Divide and Conquer (Part II)

Multiplication of two numbersLet U = (u2n-1u2n-2 … u1u0)2 and V = (v2n-1v2n-2…v1v0)2, and

our goal is to find U times V.

Ordinary multiplication requires execution time α (n2).

Alternatively, let U = 2nU1 + U0 and V = 2nV1 + V0 where

U1 = (u2n-1u2n-2 … un)2 and U0 = (un-1un-2 … u0)2

Likewise, V1 = (v2n-1v2n-2 … vn)2 and V0 = (vn-1vn-2 … v0)2

U X V = 22nU1V1 + 2n(U1V0 + U0V1) + U0V0.

Apparently there is no saving, this multiplication also requires execution time proportional to n2.

05/04/23


Multiplication: Divide and ConquerU X V = (22n+ 2n)U1V1 + 2n(U1 – U0 )( V0 – V1) + (2n+ 1)

U0V0.

which requires only three multiplications and some extra addition.

If T(n) denotes time to multiply two binary integers of size n each, then

T(2n) = 3 T(n) + cn, and T(1) =1,

where cn denotes the cost associated with additions and shifting of binary integers etc..

05/04/23


Multiplication: Divide and ConquerSolution of this recurrence equation is (by Master Theorem) or

repeated substitution, for n = 2m, is

T(2m) = 3mT(1) + c(3m-2m)

In general,

T(n) 3lg n T(1) + c(3lg n - 2lg n )

3c X 3lg n

= 3c X nlg 3 3c X 1.585

05/04/23


Matrix MultiplicationLet C = A B = (cij) be the product of two n x n matrices A and B,

where

a11 a12 … a1n b11 b12 … b1n

A = a21 a22 … a2n and B = b21 b22 … b2n

… … … … … … … …

an1 an2 … ann bn1 bn2 … bnn

• We calculate n2 terms, cij for i =1,2,…,n, , j =1,2,…,n.

• Each cij requires n scalar multiplications.• Therefore, total complexity of this matrix multiplication is of the

order n3.

05/04/23


Suppose n = 2k for some integer value of k. Letr … s a … b e … g

C = … … … = A X B = … … … X … … …

t … u c … d f … h

Where r,s,t,u are all n/2 X n/2 matrices

• In this way of obtaining C, by first partitioning A and B in submatrices, we calculate 4 matrices r,s,t, and u.

05/04/23


Suppose n = 2k for some integer value of k. Let

• Each submatrix has two matrix multiplications, of size n/2 X n/2 each.

Hence, the recurrence equation associated with this divide and conquer approach would satisfy:

T(n) = 8T(n/2) + (n2)

T(n) =(n3)

Thus, there is no saving.

05/04/23


Matrix Multiplication: Strassen’s Algorithm

P1 = a (g-h) = ag - ah

P2 = (a+b)h = ah + bh

P3 = (c+d) e = ce + de

P4 = d (f-e) = df - de

P5 = (a+d)(e+h)= ae + ah + de + dh

P6 = (b-d)(f+h) = bf + bh - df – dh

and

P7 = (a-c) (e+g) = ae + ag – ce - cg

05/04/23


Then, we can obtain r,s,t, and u as follows:r = P5+P4 - P2+P6

s = P1 + P2

t = P3 +P4

u = P5+P1 - P3-P7

For example,

P5+P4 - P2+P6

= (ae+ah+de+dh)+(df - de) - (ah + bh) + (bf + bh - df -dh)= ae+bf= r

05/04/23


Hence, by Master Theorem,

T(n) = 7T(n/2) + (n2)

= (nlg 7)

= (n2.81)

In other words, it is possible to multiply two matrices at a rate faster than (n3).

05/04/23


05/04/23

27 99 0 8 13 64 86 16 7

i x j

27 99 0 8 13 64 86 16 7

i j 7 99 0 8 13 64 86 16 27

i j

7 99 0 8 13 64 86 16 27

i j

7 16 0 8 13 64 86 99 27

i j

7 16 0 8 13 64 86 99 27

i j


Method 2:

1. Choose x = A[1].

2. Set i 2, and set j n.

3. If A[j] > x, then j j -1, else go to step 4.

4. If A[i] < x, then i i +1, else go to step 5.

5. If i < j, then interchange A[i] and A[j], otherwise set q j and interchange A[1] and A[q].

05/04/23


05/04/23

27 99 0 8 13 64 86 16 7

i x j

27 7 0 8 13 64 86 16 99

i j

27 7 0 8 13 16 86 64 99

x j i

16 7 0 8 13 27 86 64 99

x


Method 3:

1. To partition A[1],…,A[n] this method proceeds as follows:

2. Set i =1, j = n, X = A[1]

3. Compare A[i] and A[j], and exchange if A[i] A[j].

4. If no exchange is required then decrease j by 1 and continue. If exchange is required then increase i by 1 after the exchange and continue. If i = j, then stop, else

5. Go to step 2.

To see how this method works suppose we wish to partition 503, 087, 512, 061, 908, 170, and 897.

05/04/23


05/04/23

503 087 512 061 908 170 897

x,i j After 1st interchange -

170 087 512 061 908 503 897

i j After 2nd interchange -

170 087 503 061 908 512 897

i j

After 3rd interchange -

170 087 061 503 908 512 897

i j Since i = j, the algorithm stops to give the following partition 170, 087, 061, 503, 908, 512, 897 left sub-aray x right sub-array


Quick-Sort A[1], … , A[n] algorithm is:

1. Find a partition of A[1], … , A[n] such that A[1], … , A[q] are all smaller than (new) A[q+1], … , A[n] for 1 q (n -1)

2. Quick-Sort A[1], … ,A[q].

3. Quick-Sort A[q + 1], … , A[n].

05/04/23


Comments:

1. All comparisons are made with the same element. Amount of data movement is reasonable. This makes these partition procedures reasonable for large values of n.

2. Other choices for X are A[n], A[ n/2 ] and randomly chosen value i.e., A[i].

3. It makes sense to sort an array by special procedures such as Bubble-Sort, for small values of n, say n M. Some analyses and other supporting arguments suggest that M 9 is a reasonable bound for shifting from Quick-Sort to Bubble-Sort.

05/04/23


Analysis of Quick-sort:

• Exact analysis of Quick-sort is not possible --- we cannot control the size of the sub-arrays.

• We will consider the average analysis of Quick-sort.

Assumptions:

• A[1], … , A[n] are all distinct.

• A[i]'s are randomly distributed.

• A[i] {1,2, … , n} for i = 1, … , n.

• The partition Method #3 is used.

05/04/23


Observations:

1. All n! permutations of A[1], … , A[n] are equally likely.

2. Suppose A[i] = k. Then after one application of the partition method, size of the left sub-array is (k - 1),

3. size of the right sub-array is (n - k) and k is in its correct position.

4. Pr (A[i] = k) = 1/n.

5. Exactly (n + 1) comparisons are made in arranging the given array A[1], …, A[n] in partition method 2. [Reason - Each time i is increased by 1 or j is decreased by 1 a comparison is made until i becomes equal to j.]

05/04/23


05/04/23

Suppose AC(n) = Average number of comparisons required to sort A[1], … , A[n] by Quick-sort.

Then,

AC(n) = (n + 1) + (1/n){ [AC(k - 1) + AC(n - k)]}

= (n + 1) + (1/n){ [AC(k) + AC(n - k - 1)]}

= (n + 1) + AC(k)

Consequently,

n AC(n) = n (n + 1) + 2 [AC(0) + … + AC(n - 1)].

Substituting (n + 1) for n in the above equation gives,

(n + 1) AC(n + 1)

= (n + 1) (n + 2) + 2 [AC(0) + … + AC(n - 1) + AC(n)],

n

k 1

1

0

n

k

1

0

2 n

kn


05/04/23

)1(1

22

2)1(

,givesonsubstitutiRepated

.)(2

2)1(

,Then.)2(

)1()1(Let

.)1()(

22

)2()1(

or,),()2()1(2)1()1(

ly,Consequent).(2)1(2)()1()1(

:istwoabovetheofdifferencetheand

ngnn

ng

ngn

ng

nnACng

nnAC

nnnAC

nACnnnACn

nACnnnACnACn


05/04/23

16

112

121

312

33

41

11

212)1(

Hence,.1)2(

therefore,3)2(,0)1(,1)0(But

)2(422

12

22

)2(21

22

2

2

2

3

3

3

)2(

n

n

H

H

nnng

g

ACACAC

gnnn

ngnnn

AC


05/04/23

.log)(,logSince

13812

382)1(

16

112)1()(

Finally,

1

1

1

nnnACnH

nHn

Hn

HnnAC

n

n

n

n


Selection Problem

05/04/23

Find the i-th smallest element of a given array A[1], A[2], …,A[n]}

The algorithm described below borrows an idea from Quick-Sort---how to partition an array into two subarrays.

Algorithm1. Call A[1] the pivotal element.

2. Use a partition procedure, as in Quick-Sort, to find the location of the pivotal element, k, in the partitioned array (in which all elements to the left of the pivotal element are smaller than it and to the right are larger than it).

3. If i = k, then we have found the i-th smallest element; the answer is (old) A[1].


05/04/23

4. If 1 i k-1, then find the i-th smallest element among (new) A[1], … , A[k-1], else

5. if k<i n, then find the i-th smallest element among new A[k+1], … ,A[n]. (This case can be seen as equivalent to finding the (i-k)-th smallest element out of (n-k) of the right subarray after renaming A[k+1], … , A[n] as A[1], … ,A[n-k]).

Analysis of the above algorithm

Please see the book

Divide and Conquer

05/04/23

Questions

divide and conquer (part ii) multiplication of two numbers let u = (u 2n-1 u 2n-2 … u 1 u 0 ) 2...

Documents