fast fourier transform: theory and algorithms · pdf filegeneralizations the inner-most sum...

Fast Fourier Transform: Theory and Algorithms

Lecture 8

6.973 Communication System Design – Spring 2006

Massachusetts Institute of Technology

Vladimir Stojanović

Cite as: Vladimir Stojanovic, course materials for 6.973 Communication System Design, Spring 2006.MIT OpenCourseWare (http://ocw.mit.edu/), Massachusetts Institute of Technology.

Downloaded on [DD Month YYYY].

Discrete Fourier Transform – A review

� Definition

� {Xk} is periodic � Since {Xk} is sampled, {xn} must also be periodic

� From a physical point of view, both are repeated with period N

� Requires O(N2) operations

6.973 Communication System Design 2 Cite as: Vladimir Stojanovic, course materials for 6.973 Communication System Design, Spring 2006.

MIT OpenCourseWare (http://ocw.mit.edu/), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Fast Fourier Transform History

Twiddle factor FFTs (non-coprime sub-lengths) �

� 1805 Gauss � Predates even Fourier’s work on transforms!

� 1903 Runge

� 1965 Cooley-Tukey

� 1984 Duhamel-Vetterli (split-radix FFT)� FFTs w/o twiddle factors (coprime sub-lengths)

� 1960 Good’s mapping � application of Chinese Remainder Theorem ~100 A.D.

� 1976 Rader – prime length FFT � 1976 Winograd’s Fourier Transform (WFTA) � 1977 Kolba and Parks (Prime Factor Algorithm – PFA)



Divide and conquer

Suppose all Il sets have same number of elements N1 so, N=N1*N2, r=N2

Each inner-most sum takes N12 multiplications

The outer sum will need N2 multiplications per output point

� Hence, total number of multiplications

� Divide and conquer always has less computations

N2*N for the whole sum (for all output points)

6.973 Communication System Design 4



Generalizations

� The inner-most sum has to represent a DFT � Only possible if the subset (possibly permuted)

� Has the same periodicity as the initial sequence � All main classes of FFTs can be cast in the above form

� Both sums have same periodicity (Good’s mapping) � No permutations (i.e. twiddle factors) � All the subsets have same number of elements m=N/r

� (m,r)=1 – i.e. m and r are coprime

� If not, then inner sum is one stap of radix-r FFT � If r=3, subsets with N/2, N/4 and N/4 elements

� Split-radix algorithm



The cost of mapping

� The goal for divide and conquer

� Different types balance mapping with subproblem cost

� E.g. in radix-2 � subproblems are trivial (only sum and differences) � Mapping requires twiddle factors (large number of

multiplies) � E.g. in prime-factor algorithm

� Subproblems are DFTs with coprime lengths (costly) � Mapping trivial (no arithmetic operations)



FFTs with twiddle factors� Reintroduced by Cooley-Tukey ‘65

Start from general divide and conquer

Keep periodicity compatible with periodicity of the input sequence Use decimation

almost N1 DFTs of size N2



Cooley-Tukey FFT contd.

can be taken mod N2

1 1 2, ,n k n kY Y=

2. N multplications with twiddle factors

1. N1 DFTs of length N2

3. N2 DFTs of length N1

6.973 Communication System Design 8Cite as: Vladimir Stojanovic, course materials for 6.973 Communication System Design, Spring 2006.


� Step 1: Evaluate N1 DFTs of length N2

� Step 2: N multiplications with twiddle factors � Step 3: Evaluate N2 DFTs of length N1

� Vector xi mapped to matrix xn1,n2 (N1xN2) � Compute N1 DFTs of length N2 on each row � Point-to-point multiply with twiddle factors � Compute N2 DFTs of length N1 on the columns



2-D view of Cooley-Tukey mapping

� N=15 (N1=3, N2=5)

� Cannot exchange the order of DFTs � Because of twiddle multiply � Different mapping for N1=5, N2=3

� Not just transpose 6.973 Communication System Design 10



DFT-3

DFT-3

x0

x0

x0

x0

x0

x3

x3

x3

x6

x6

x6

x9

x9

x9

x9

x12

x12

x12

x12

x1

x1

x1...............

x1

x4

x4

x4

x4

x7

x7

x10

x10

x10

x13

x13

x13

x13

x2

x2

x2

x5

x5

x5x5

x8

x8

x11

x11

x11

x14

x14

x14

x14

DFT-5

DFT-5

DFT-5

jm

DFT-3

DFT-3

DFT-3

W

x0 x5 x10

x1 x6 x11

x2 x7 x12

x3 x8 x13

x4 x x9 14

Figure by MIT OpenCourseWare.


Radix 2 and radix 4 algorithms

� Lengths as powers of 2 or 4 are most popular � Assume N=2n

� N1=2, N2=2n-1 (divides input sequence into even and odd samples – decimation in time – DIT)

“Butterfly”(sum or difference followed or preceeded by a twiddle factor multiply)

� Xm and XN/2+m outputs of N/2 2-pt DFTs on outputs of 2, N/2-pt DFTs weighted with twiddle factors



DIT radix-2 implementations

� Several different ways � Reorder the input data

� Input samples for inner DFTs in subsequent locations

� Results in bit-reversed input, in-order output DIT

� Selectively computeDFTs on evens and odds � Perform in-place

computation � Output in bit-reversed

order (X3 in position six (011->110))

6.973 Communication System Design



12

DFT

N = 2

DFT

N = 2

DFT

N = 2

DFT

N = 2

X0X0

X4

X4

X1

X1

X5

X5

X2

X2

X6

X6 X3

X3

X7X7

DFT

N = 4

{ }

w18

w28

w38

Divisioninto even and odd numbered

sequences

DFT ofN / 2

Multiplicationby twiddle

factors

DFT of2

DFT

N = 4

{ }x2i

x2i+1

Which type is this implementation? Figure by MIT OpenCourseWare.

Decimation in frequency (DIF) radix-2 implementation

� If reverse the role of N1 and N2, get DIF� N1=N/2, N2=2



DFTN = 4

DFTN = 2

DFTN = 2

DFTN = 2

DFTN = 2

X0 X0

X4

X4

X1

X1

X5

X5

X2

X2

X6

X6X3

X3

X7 X7

w18

w28

w38

DFT of2

DFT ofN / 2


factors

{ }x2k

DFTN = 4

{ }x2k+1

n1=0

n1=0

=

=

X2k1

X2k1+1

N / 2-1

N / 2-1

N / 2(xn1+xN / 2+n1

),

NN / 2 Wn1(xn1

-xN / 2+n1).

n1k1

n1k1

W

W


Duality DIT<->DIF

6.973 Communication System Design Cite as: Vladimir Stojanovic, course materials for 6.973 Communication System Design, Spring 2006.


14

DFT

N = 2

DFT

N = 2

DFT

N = 2

DFT

N = 2

X0

X4

X1

X5

X2

X6

X3

X0

X4

X1

X5

X2

X6

X3

X7X7

w18

w28

w38

Divisioninto even and

odd numberedsequences

DFT ofN / 2


factors

DFT of2

DFT

N = 4

DFT

N = 2

DFT

N = 2

DFT

N = 2

DFT

N = 2

X0 X0

X4

X4

X1

X1

X5

X5

X2

X2

X6

X6X3

X3

X7 X7

DFT of2

DFT ofN / 2


factors

{ }x2k

DFT

N = 4

{ }x2k+1

DFT

N = 4

{ }x2i

DFT

N = 4

{ }x2i+1

w18

w28

w38

� Which one is DIT (DIF)? � How can we get one from another?

Figure by MIT OpenCourseW. are.

Complexity of radix-2 FFTs

� DFT of length N replaced by two length-N/2 � At the cost of N complex multiplications (twiddle)

� And N complex additions (2pt DFTs)

� Iterate the scheme log2N-1 times � Obtain trivial transforms (length 2) of the length-N/2 DFTs

� Twiddle multiplies (WNi)

� Complex multiply – 3 real mult + 3 real add � If i is multiple of N/4, no arithmetic operation required (why?)

4 butterflies (one general, 3 special cases)



Radix-4

� N=4n, N1=4, N2=N/4 � 4 DFTs of length N/4 � 3N/4 twiddle multiplies � N/4 DFTs of length 4

� Cost of length-4 DFT � No multiplication � Only 16 real additions

� Reduces the number of stages to log4N

� Radix-8 can reduce number of operations even more 6.973 Communication System Design 1



6

DFT8

DFT8

DFT2

DFT4

DFT4

DFT4

DFT4

DFT4

X15

X1

X14

X0x0

X0x0

X12

x12

X1

x8

X13

X14

x4

X2

X15x15

X3

x8

x15


Mixed-radix and Split-radix

� Mixed-radix � Diferent radices in different

stages � Split-radix

� Different radices in the same stage � Simultaneously on different

parts of the transform

� Can achieve lowest number of adds and multiplies for length 2n inputs



17



R2

R4

S-R

x0 X0

x4

x8

x12

X14X1

X13X3

X15

DFT2/4

DFT8

DFT4

DFT4

DFT8

DFT8

DFT2

DFT4

DFT4

DFT4

DFT4

DFT4

X15

X1

X14

X0x0

X0x0

X12

x12

X1

x8

X13

X14

x4

X2

X15x15

X3

x8

x15


Split-radix (DIF SRFFT)

DFT2/4

X0 X0

X4

X14X1

X13X3

X15

X8

X12

X[(xn1+xN / 2+n1

)

-j(xn1+N / 4-xn1+3N / 4)].

DFT8

DFT4

DFT4

DFT

N = 4

DFT

N = 2

DFT

N = 2

DFT

N = 2

DFT

N = 2

X0 X0

X4

X4

X1

X1

X5

X5

X2

X2

X6

X6X3

X3

X7 X7

w18

w28

w38

DFT of2

DFT ofN / 2


{ }x2k

DFT

N = 4

{ }x2k+1

n1=0=X2k1

N / 2-1N / 2(xn1

+xN / 2+n1),

X[(xn1-xN / 2+n1

)

+j(xn1+N / 4-xn1+3N / 4)],

n1k1W

n1=0=X4k1+1

N / 4-1

NN / 4Wn1n1k1W

n1=0=X4k1+3

N / 4-1

NN / 4W3n1n1k1W

n1=0

n1=0

=

=

X2k1

X2k1+1

N / 2-1

N / 2-1

N / 2(xn1+xN / 2+n1

),

NN / 2 Wn1(xn1

-xN / 2+n1).

n1k1

n1k1

W

W

factors

� Even samples X2k in DIF should be computed separately from other samples � With same algorithm (recursively) as

the original sequence

� No general rule for odd samples � Radix-4 is more efficient than radix-2 � Higher radices are inefficient



� Look at DIF radix-2 � X don’t have twiddle2k1 s



Split-radix (DIT SRFFT)� Dual to DIF SRFFT

� Considers separately subsets {x2i}, {x4i+1} and {x4i+3}

� Redundancy in Xk, Xk+N/4, Xk+N/2, Xk+3N/4 computation



FFTs without twiddle factors

� Divide and conquer requirements � N-long DFT computed from DFTs with lengths that are

factors of N (allows the inner sum to be a DFT) � Provided that subsets Il guarantee periodic xi

� When N factors into co-prime factors N=N1*N2 � Starting from any xi form subset with compatible periodicity (the

periodicity of the subset divides the periodicity of the set) {x + n | n2 = 1,..., N2 −1} or x i N n | n1 = 1,..., N1 −1}{i N1 2 + 2 1

� Both subsets have only one common point xi

� Allows a rearrangement of the input (periodic) vector into a matrix with a periodicity in both dimensions (rows and columns), both periodicities being comatible with the initial one



Good’s mapping

�

mapping � Turns original transform into a set of small DFTs with

coprime lengths

{xi N+ n | n2 = 1 2 1 2

,..., N −1} or {xi+ N 1

| n = 1,..., 2n 1 N1 −1}

equivalent to

� This mapping is one-to-one if N1 and N2 are coprime � All congruences modulo N1 obtained

� For a given congruence modulo N2 and vice versa

FFTs without twiddle factors all based on the same



21

0

0

1

1

2

2

3

3

4

4

5

5

6

6

7

7

8

8

9

9

10

10

11

11

12

12

13

13

14

14

Good's mapping


Just another arrangement of CRT

�

� If we know the residue of some number k modulo two coprime numbers N1 and N2 k

Nk

1 N2

� It is possible to reconstruct k N N1 2

�

N

k N = k1 k

N = 2

Then 1 2

k N

k

� = t k + N N 1 1 2

N2 2 1

1 2

t1 1N = 1 and t N N 2 2 =

2 N 1

1

t1 multiplicative inverse of N1 mod N2t2 multiplicative inverse of N2 mod N1t1, t2 always exist since N1, N2 coprime (N1,N2)=1

What are t1, t2 for N1=3, N2=5?

� Reversing N1 and N2� Results in transposed mapping

6.973Cite as: Vladimir Stojanovic, course materials for 6.973 Communication System Design, Spring 2006.


Chinese Remainder Theorem (CRT)

Let t k 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

Good's mapping

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

CRT mapping

22


Impact on DFT

� Formulating the true multi-dimensional transform k = N t k + N t k

N N 1 1 2 2 2 1 1 2 N



23

Xk = Σ xi WN, k = 0,..., N - 1,i = 0

ik

XN1t1k2 + N2r2k1 = Σ Σ xn1N2 + n2N1 WN

N2 - 1N1 - 1

n2 = 0n1 = 0

(n1N2 + N1n2)(N1t1k2+N2t2k1)

WN = WN1

N2 WN1 = WN1 = WN1

N2t2 (N2t2)N1

XN1t1k2 + N2t2k1 = Σ Σ xn1N2 + n2N1 WN1 WN2

N2 - 1N1 - 1

n2 = 0n1 = 0

n1k2 n2k2 ,

X'k1k2 = XN1t1k2 + N2t2k1

x'n1.n2 = xn1N2 + n2N1

X'k1k2 = Σ Σ x'n1n2 WN1 WN2

n2k2 n1k1N2 - 1N1 - 1

n2 = 0n1 = 0

DFT3

DFT5

X9

X4

X14X8

X2

X11

X5

x12

x9

x6

x3

x0

x5

x10

N - 1

Figure by MIT OCW.

True bidimensional transform!(no extra twiddle factors)


Using convolution to compute DFTs

� All sub DFTs are prime length� Rader showed that prime-length DFTs can be

computed as a result of cyclic convolution

� E.g. length 5 DFT

Permute last two rows and columns

Cyclic correlation (a convolution with a reversed sequence)

� This is a general result



Example

� Results in smallest number of multiplies



Prime Factor Algorithm

� Efficient small DFTs are a key to the feasibility of this algorithm



26

X = F1 x F

2T.

x12

x9

X9

X4

X14

X8

X2

X11

X5

x6

x3

x0

x5

x10

DFT5

DFT3

Good's mapping CRT mapping


Winograd’s Fourier Transform Algorithm

� B1xB2’ only involves additions � D – diagonal (so point multiply) � Winograd transform has many more additions than

twiddle FFTs



27

x10

x5x0

x3

x6

x9

X9

X4

X14

X8

X2

X11

X5

x12

input additions input additions output additions output additionspoint wise multiplication N = 3N = 5N = 5N = 3


fast fourier transform: theory and algorithms · pdf filegeneralizations the inner-most sum...

Documents