instructor neelima gupta [email protected]. table of contents divide and conquer

InstructorNeelima Gupta

[email protected]

Table of Contents

Divide and Conquer

Divide the problem into smaller sub-problems.

Solve the sub-problems recursivelyCombine the solution of the smaller sub-problems to obtain the solution of the bigger problem.

Size of subproblems must reduce There must be some initial conditions

The two together ensures that the algorithm terminates

Familiar Egs of divide and conquer

Quick sort Merge sort (John von Neumann, 1945)

Time

If T(n) is the time to sort ‘n’ numbers :- T(n)=T(n/2)+T(n/2)+Ө(n) =2T(n/2) + cn =Ө(nlog n)Where Ө(n )= time required to merge two sorted lists, each of size at most n/2

Multiplying 2 large integersSuppose for simplicity, numbers are of equal length Eg-

suppose the 2 number are 2345 and 3854

O(n2) time by the usual successive add algorithm.

We can reduce this time using divide and conquer strategy.

Multiplying 2 large integers

split the numbers into roughly 2 halves

Here x0 =45; x1 =23; y0 =54; y1 =38;

Mathematically x= x1 .10n/2 + x0 (1)

y= y1 .10n/2 + y0 (2)

xy=x1y1.10n+(x1y0+y1x0).10n/2+x0y0 (3)

In our eg n=4 x=23*102 + 45 y=38*102 + 54

x1y1=874;x1y0=1242;x0y1=1710;x0y0=2430;

Using (1),(2),(3) answer came out to be

= 874*104 +(2952)*102+2430 adding these numbers take linear time.

Time AnalysisComputing x1y1,(x1y0+y1x0),x0y0

T(n)=4T(n/2)+cn =Ө(n2)

So, no improvement. We’ll use a smarter way to compute the middle term.

(x0+x1)(y0+y1)=x1y1+(x1y0+y1x0)+x0y0

Thus,x1y0+y1x0 = (x0+x1)(y0+y1) - x0y0 - x1y1

Thus, only 3 multiplications suffice

So T(n)is reduced to T(n)=3T(n/2)+cn =Ө(nlog

23)<Ө(n2)

where 3T(n/2) is the time required to multiply

x1y1 , (x0+x1)(y0+y1), x0y0

Complex Roots of UnityConsider xn = 1

It has n distinct complex roots as follows:

ωj,n = e (2 π ij)/n , j = 1 to n – 1

where i = √-1

These numbers are called nth roots of unity

nth roots of unityFor example, for n = 2 x 2 = 1 => x = +1 , -1

e (2 π ij)/n , j = 0,1

j = 0 e (2 π ij)/n = e0 = cos 0 + i sin 0

= 1

j = 1 e (2 π i)/2 = eπi = cos π + i sin π = -1

Thanks to : Megha and Mitul

These can be pictured as a set of equally spaced points lying on a unit circle as shown in figure for n = 8.

Figure by Mitul

If ωj,2n ( j = 0, 1 … 2n-1) are (2n)th roots of

unity then clearly ω2j,2n ( j = 0, 1 … n-1) are

nth roots of unity. For j = n … 2n -1, roots repeat.

ω j,2n = e (2 π ij)/2n = e (π ij)/n ω j,2n

2 = (e (π ij)/n )2 = e (2 π

ij)/n It is also visible from the figure below

Figure by Mitul

Discrete Fourier TransformDFT of a polynomial with coefficient vector <a0,a1,….,an> is the vector y = <y0,y1,….,yn> where

Θ(n2) time to compute DFT is trivial, each yi can be computed in Θ(n) time.

FFT is a method to compute DFT in Θ(n log n) time, It makes use of special properties of complex roots of unity.

n

ij

j eAy2

Fast Fourier TransformInstead of n, we will deal with 2n

Break the polynomial in 2 parts : Aeven(x) = a0 + a2x + … + an-2

x (n-2)/2

Aodd(x) = a1 + a3x + a5x2 + ….+ an-1x(n-2)/2

A(x) = Aeven(x2) + x.Aodd(x2)

Let

p(x) = Aeven(x2) = a0+a2x2 + a4x4+….+ an-2xn-2 and

q(x) = Aodd(x2) + a1 + a3x2 + a5x4+….+an-1xn-2

x.q(x) = a1x + a3x3 + a5x5+…+an-1xn-1


Fast Fourier Transform contd…Thus,

yj = A(ωj,2n ) = Aeven(ω2j,2n ) +

ωj,2n .Aodd(ω2j,2n )

(2n)th root of unity nth root of unity

Aeven and Aodd are polynomials with n terms, thus they can be computed at nth roots of unity, recursively.

FFT contd..Thus we arrive at the following recurrence to compute the DFT

T(n) = 2T(n/2) + O(n) = n log n

Thus given a polynomial, its DFT can be computed in O(n log n) time.

Vandermonde MatrixComputing DFT is equivalent to

n is only a control variable and can be replaced by 2n

Since DFT can be computed in O(n log n) time, this matrix-vector product can be computed in O(n log n) time.

.

...

...

1......

...

......

11

1...11

...

...

1

1

0

11,1

11,2

11,1

2,1

2,1

,1,2,1

1

1

0

nn

nnnn

nn

nnn

nnnn

na

a

a

y

y

y

Vandermonde matrix

Computing DFT-1

Theorem:For j,k = 0 …2n -1, (j,k) entry of V-12n

is ω-kj,2n /2n

Thus, given yi ‘s , ai ‘s and hence the

polynomial can be computed as follows:

As before n can be replaced by 2n. This matrix-vector multiplication is similar to the previous one and hence can be computed in O(n log n) time.

.

...

...

...1

...............

......1

...1

11...11

...

...

1

1

0

)1(,1

)1(,2

)1(,1

2,1

2,1

1,1

1,2

1,1

1

1

0

nnnn

nn

nn

nnn

nnnn

ny

y

y

nnn

nn

nnn

a

a

a

Convolution

Let A = <a0,a1,….,an> and

B = <b0,b1,…..,bn> be 2 vectors

C = A o B = <c0,c1,…..,c2n-2> (Length = 2n -1)

C k = ∑ i+j = k aibj

AIM : to obtain convolution in (nlog(n)) time


Applications: polynomial multiplication

Suppose we have two polynomials

A(x) = a0 + a1x + ………+ an-1xn-1

B(x) = b0 + b1x + ………+ bn-1xn-1

Then, C(x) = A(x).B(x)

Algortihm for AoBLet A(x) and B(x) be two polynomials with coefficients from the vectors A and B respectively.

1. Compute A and B at (2n)th roots of unity i.e. compute A(ωj,2n ) and B(ωj,2n ) , j = 0,1 …2n-1 using FFT.

O(n log n) time.2. Compute C(ωj,2n ) = A(ωj,2n ) . B(ωj,2n ) …. O(n) time.3. Reconstruct C(x) using FFT-1 ….O(n log n) time.

Submitted By:

Jewel Pruthi(18) Juhi

Jain(19)

Problem : Given a set of points p1,p2,--------pn,

our aim is to find out the closest pair

of points.

Finding Closest Pair in one-dimension Complexity – O(nlogn) How?

Thanks to Jewel and Juhi

Sort the points.(takes O(nlogn) time)Find distance between every pair of consecutive points.

In n comparisons,we will find the distance between every pair of consecutive points.

In another n comparisons,we will find minimum of the distances found.


Finding Closest Pair in two-dimensionProposed algorithm:Sort the points p1,p2,------pn say on increasing order of x-coordinates.

where pi=(xi,yi)

Distance between 2 consecutive points d=sqrt((y2-y1)2 + (x2-x1)2 )

st x1≤x2≤x3------≤xn

Ques: Will this algorithm work?


No

Consider p1,p2,p3 st d(p1p2) > d(p2p3) after sorting p1,p2,p3 on x-coordinates.

Algorithm returns p2p3 but we can see that p1p3 are closer.


p1

p2

p3

Note : For minimum distance,the two points need not be consecutive on x/y-axis.

Brute Force Approach Calculate distance between every possible pair of points.

Time Complexity:- O(n2)

Can we improve on the time complexity?


Divide and Conquer ApproachArrange the points in increasing order of x-coordinates say Px & increasing order of y-coordinates say Py.

Divide the set of n points into 2 halves Q and R (breaking on middle of Px).

Compute the closest pair in Q and in R recursively.


Solve recursively

Solve recursively

Q R

Let (q1,q2) and (r1,r2) be the closest pairs obtained in Q and R respectively.

Let δ=min { d(q1,q2) , d(r1,r2) }


Solve recursively

Solve recursively

Q R

(q1,q2) (r1,r2)

Ques: Does Ǝ a pair of points say (s1,s2) such that d(s1,s2) < δ ?

Soln: Let p* be the point with maximum x-coordinate in Q and let x* be its x-coordinate.

Draw a line through p* described by the equation L : x=x*



p*

δ

δ

L

x*

q(qx)

r(rx)

Q

RThis distance is x*-qx

Now in the highlighted triangle we can see that the length

of horizontal line (x*-qx) < hypotenuse according to

Pythagoras theorem.x*-qx<hypotenuse<d(q,r)< δ

Similarly we can prove that rx-x* < d(q,r) < δ

Consider square boxes each of side δ/2 in this vertical strip.


δ/2

δ/2

S

qy

ry

L

Square of side δ/2

Claim : No box contains more than 1 point.

Proof: If the 2 farthest points in the box are the 2 diagonal points of the square, then

1.Distance between the 2 points =

√( (δ/2)2 + (δ/2)2 ) = (δ/√2) < δ

2. Both the points are within Q or both are within R

This implies that we have two points within Q (/R) with distance < δ which is a contradiction to the definition of δ.


Claim : Between any pair of 2 points q and r in S with d(q,r) < δ , there can be no more than 12 points.


No of points ≤ 12

ṕ1

ṕ2

ṕk

ṕr

Proof: Let Py1 … Pyt be the points of S in the increasing order of y co-ordinates.

If there are more than 12 points between q and r in the above list then there will be at least 3 rows between the y co-ordinate of q and y co-ordinate of r.

Then, the vertical distance between q and r is > 3 × (δ/2) > δThus, the actual distance which is > vertical distance (show through fig.) > δ

---Contradiction to the definition of points q and r.


For i = 1 to tFor j = i+1 to i + 12

Compute d(Pyi , Pyj)

Compute the minimum of the 12t pairs above --- O(n) time.

THANKYOU

instructor neelima gupta [email protected]. table of contents divide and conquer

Documents