16 fft

Comp 750, Fall 2009 FFT - 1Jim Anderson

Chapter 30: Polynomials and the FFT

1n

0j

jjxaA(x)Polynomial:

coefficient: complex, e.g.,3.25 + 6.999i

real part imaginary part-1

Degree-bound is n.

Degree is k if ak is the highest nonzero coefficient.

Comp 750, Fall 2009 FFT - 2Jim Anderson

Polynomial Addition

1n

0j

jjxaA(x)

1n

0j

jjxbB(x)

jjj

1n

0j

jj bac ,xcB(x)A(x)C(x)

(n) time to compute.

Comp 750, Fall 2009 FFT - 3Jim Anderson

Polynomial Multiplication

j

0kkjkj

22n

0j

jj

bac where

xc

B(x)A(x)C(x)

called convolution

Note: Degree bound of C(x) is 2n – 1.

Straightforward computation takes (n2) time.

Comp 750, Fall 2009 FFT - 4Jim Anderson

Example

6x3 + 7x2 – 10x + 9 –2x3 + 4x – 5 –30x3 – 35x2 + 50x – 45 24x4 + 28x3 – 40x2 + 36x–12x6 – 14x5 + 20x4 – 18x3 –12x6 – 14x5 + 44x4 – 20x3 – 75x2 + 86x – 45

Comp 750, Fall 2009 FFT - 5Jim Anderson

Faster Multiplication

Using the FFT, can multiply two polynomials in (n lg n) time.

Idea: Represent polynomials in a way that allows fastermultiplication.

1. Convert to new representation.

2. Multiply in (n) time.

3. Convert back.

Want these steps to run in (n lg n) time.

Comp 750, Fall 2009 FFT - 6Jim Anderson

Representation of PolynomialsCoefficient Representation:

Advantages:

1) Can evaluate at any point x0 in (n) time using Horner’s Rule:

A(x0) = a0 + x0(a1 + x0(a2 + … + x0(an-2 + x0(an-1)) … ))

2) Can add in (n) time.

Multiplication is a problem.

Point-Value Representation: {(x0, y0), (x1, y1), …, (xn-1, yn-1)}, where xk’s are distinct and yk = A(xk).

Converting from coefficient to P-V using Horner’s Rule takes(n2) time. We want to do it in (n lg n) time.

1n

0j

jjxaA(x)

Comp 750, Fall 2009 FFT - 7Jim Anderson

Addition and Multiplication in P-VAddition: C(x) = A(x) + B(x)

A: {(x0, y0), (x1, y1), …, (xn-1, yn-1)} B: {(x0, y0), (x1, y1), …, (xn-1, yn-1)} C: {(x0, y0 + y0), (x1, y1 + y1), …, (xn-1, yn-1 + yn-1)}

Can compute C in in (n) time.

Multiplication: C(x) = A(x) B(x)

Must “extend” A and B to 2n P-V pairs:

A: {(x0, y0), (x1, y1), …, (x2n-1, y2n-1)} B: {(x0, y0), (x1, y1), …, (x2n-1, y2n-1)} C: {(x0, y0 y0), (x1, y1 y1), …, (x2n-1, y2n-1 y2n-1)}

Also takes (n) time.

Actually need only2n–1 pairs, but weassume 2n forsimplicity.

Comp 750, Fall 2009 FFT - 8Jim Anderson

Strategy for Multiplying Polynomialsa0, a1, …, an-1

b0, b1, …, bn-1

c0, c1, …, c2n-2

A(2n0), B(2n

0)A(2n

1), B(2n1)

A(2n

2n-1), B(2n2n-1)

C(2n0)

C(2n1)

C(2n

2n-1)

Ordinary multiplication

(n2) time

Evaluation(n lg n) time

Interpolation(n lg n) time

Pointwise multiplication

(n) time

Coeff.Rep.

P.V.Rep.

Note: Powerpoint doesn’t do a great job with combinationsubscripts/superscripts. See book for how this is supposed to look.

The “n” you see in subsequent slides corresponds to “2n” here.

Comp 750, Fall 2009 FFT - 9Jim Anderson

The DFT and FFT

We convert from coeff. to PV by evaluating at the n complex nth

roots of unity.

Denote as n0, n

1, …, nn-1.

nk = e2ik/n.

We have (nk )n = 1 for each k.

Proof:(n

k )n = e2ik

= cos(2k) + i sin(2k), follows from eiu = cos(u) + i sin(u). = 1

Principle nth root of unity: n = e2i/n.Other roots are powers of this one.

Comp 750, Fall 2009 FFT - 10Jim Anderson

Example

1–1

–i

i

87

86

85

84

83

82

81

80 = 8

8

The values of 80, 8

1, …, 87 in the complex plane, where

8 = e2i/8 is the principle 8th root of unity.

Comp 750, Fall 2009 FFT - 11Jim Anderson

Cancellation Lemma

Lemma 30.3: For all n 0, k 0, and d > 0: dndk = n

k.Lemma 30.3: For all n 0, k 0, and d > 0: dndk = n

k.

Proof:

dndk = (e2i/dn)dk

= (e2i/n) k

= nk

Corollary 30.4: For all even n > 0: nn/2 = 2 = –1. Corollary 30.4: For all even n > 0: n

n/2 = 2 = –1.

Comp 750, Fall 2009 FFT - 12Jim Anderson

Halving Lemma

Lemma 30.5: If n > 0 is even, then the squares of the n complex nth

roots of unity are the n/2 complex (n/2)th roots of unity.

Lemma 30.5: If n > 0 is even, then the squares of the n complex nth

roots of unity are the n/2 complex (n/2)th roots of unity.

Proof:

(nk)2 = n/2

k

Note also: (nk+n/2)2 = n

2k+n

= n2k n

n

= n2k

= (nk)2

Thus, nk and n

k+n/2 have the same square.

Comp 750, Fall 2009 FFT - 13Jim Anderson

Summation LemmaLemma 30.6: For all n 1, k 0, where k is not divisible by n:Lemma 30.6: For all n 1, k 0, where k is not divisible by n:

.0ωj1n

0j

kn

Proof:

0

1ω

11

1ω

1ω

1ω

1ωω

kn

k

kn

knn

kn

nkn

j1n

0j

kn

Where is “k not divisibleby n” needed?

Comp 750, Fall 2009 FFT - 14Jim Anderson

The DFT

1n

0j

jjxaA(x)We wish to evaluate at n

0, n1, …, n

n-1.

W.o.l.o.g., assume n is a power of 2.

Define:

The vector y = (y0, y1, …, yn-1) is called the Discrete FourierTransform (DFT) of the coefficient vector a = (a0, a1, …, an-1).

We write: y = DFTn(a).

Our Goal: To compute y efficiently.

1n

0j

kjnj

knk ωa)A(ω y

Comp 750, Fall 2009 FFT - 15Jim Anderson

The Fast Fourier Transform (FFT)The FFT computes DFTn(a) in (n lg n) time using a divide andconquer approach.

Define: A[0](x) = a0 + a2x + a4x2 + … + an-2xn/2-1

A[1](x) = a1 + a3x + a5x2 + … + an-1xn/2-1

Both are degree-bound n/2 polynomials.Note: A(x) = A[0](x2) + xA[1](x2).

Approach:1. Evaluate A[0](x) and A[1](x) at (n

0)2, (n1

)2, …, (nn-1

)2.

2. Combine results using A(x) = A[0](x2) + xA[1](x2).

So, we solve one problem of size n by solving two problems(of the same form) of size n/2.

n/2 complex (n/2)th roots of unity (by Halving Lemma)

Comp 750, Fall 2009 FFT - 16Jim Anderson

Base Case

1n

0j

kjnj

knk ωa)A(ω y

n = 1.

Applying to , 0 k < n, yields

y0 = a010

= a0 1 = a0

So: y = (y0) = (a0) = a.

Comp 750, Fall 2009 FFT - 17Jim Anderson

CodeRecursive-FFT(a)1 n := length[a];2 if n =1 then3 return a

fi;4 n := e2i/n;5 := 1;6 a[0] := (a0, a2, …, an-2);7 a[1] := (a1, a3, …, an-1);8 y[0] := Recursive-FFT(a[0]);9 y[1] := Recursive-FFT(a[1]);10 for k := 0 to n/2 – 1 do11 yk := yk

[0] + yk[1];

12 yk+(n/2) := yk[0] - yk

[1];13 := n

od;14 return y

Recursive-FFT(a)1 n := length[a];2 if n =1 then3 return a

fi;4 n := e2i/n;5 := 1;6 a[0] := (a0, a2, …, an-2);7 a[1] := (a1, a3, …, an-1);8 y[0] := Recursive-FFT(a[0]);9 y[1] := Recursive-FFT(a[1]);10 for k := 0 to n/2 – 1 do11 yk := yk

[0] + yk[1];

12 yk+(n/2) := yk[0] - yk

[1];13 := n

od;14 return y

Time complexity:T(n) = 2T(n/2) + (n) = (n lg n)

Comp 750, Fall 2009 FFT - 18Jim Anderson

CorrectnessLines 8 and 9 compute: y[0]

k = A[0](n/2k) k = 0, …, n/2 – 1

y[1]k = A[1](n/2

k) k = 0, …, n/2 – 1

By the Cancellation Lemma: y[0]k = A[0](n

2k) y[1]

k = A[1](n2k)

For k = 0, 1, …, n/2 – 1 we get (line 11): yk = yk[0] + n

k yk[1]

= A[0](n2k) + n

k A[1](n2k)

= A(nk)

We also get (line 12): yk+(n/2) = yk[0] – n

k yk[1]

= yk[0] + n

k+(n/2) yk[1]

= A[0](n2k) + n

k+(n/2) A[1](n2k)

= A[0](n2k+n) + n

k+(n/2) A[1](n2k+n)

= A(nk+(n/2))

Comp 750, Fall 2009 FFT - 19Jim Anderson

InterpolationWe just showed we can use the FFT algorithm to evaluate

in (n lg n) time.

How do we interpolate (i.e., go from P-V back to coeff.)?

Can show

(See book need the Summation Lemma here.)

So, with minor modifications, we can use the FFT algorithm tointerpolate in (n lg n) time.

1n ..., 1, 0, k ωa y1n

0j

kjnjk

1n ..., 1, 0, k ωyn

1 a

1n

0k

kj-nkj

Comp 750, Fall 2009 FFT - 20Jim Anderson

Speeding Up the Algorithm

Can speed up the FFT code in practice by making it iterative and eliminating common subexpressions in loop. Resulting code:

Iterative-FFT(a)Bit-Reverse-Copy(a, A);n := length[a];for s := 1 to lg n do

m := 2s;m := e2i/m; := 1;for j := 0 to m/2 – 1 do

for k := j to n – 1 dot := A[k + m/2];u := A[k];A[k] := u + t;A[k + m/2] := u – t

od := m

od od;return A

Iterative-FFT(a)Bit-Reverse-Copy(a, A);n := length[a];for s := 1 to lg n do

m := 2s;m := e2i/m; := 1;for j := 0 to m/2 – 1 do

for k := j to n – 1 dot := A[k + m/2];u := A[k];A[k] := u + t;A[k + m/2] := u – t

od := m

od od;return A

Bit-Reverse-Copy(a, A)n := length[a];for k := 0 to n – 1 do

A[rev(k)] := ak

od

Bit-Reverse-Copy(a, A)n := length[a];for k := 0 to n – 1 do

A[rev(k)] := ak

od

See book for explanation ofthis code.

Comp 750, Fall 2009 FFT - 21Jim Anderson

Final Comment

In signal processing applications, FFTs are often computed in hardware.

The book gives such a hardware circuit.