fft lecture

Upload: kokoc25

Post on 08-Apr-2018

249 views

Category:

Documents


0 download

TRANSCRIPT

  • 8/6/2019 Fft Lecture

    1/90

    Fast Fourier Transform

    Key Papers in C

    Dima Batenkov

    Weizmann Institute of Science

    [email protected]

  • 8/6/2019 Fft Lecture

    2/90

    Fast Fourier Transform -

    erview

    Fast Fourier Transform - Overv

    J. W. Cooley and J. W. Tukey. An algorithmcalculation of complex Fourier series. MathComputation, 19:297301, 1965

  • 8/6/2019 Fft Lecture

    3/90

    Fast Fourier Transform -

    erview

    Fast Fourier Transform - Overv

    J. W. Cooley and J. W. Tukey. An algorithmcalculation of complex Fourier series. MathComputation, 19:297301, 1965 A fast algorithm for computing the Discre

    Transform

  • 8/6/2019 Fft Lecture

    4/90

    Fast Fourier Transform -

    erview

    Fast Fourier Transform - Overv

    J. W. Cooley and J. W. Tukey. An algorithmcalculation of complex Fourier series. MathComputation, 19:297301, 1965 A fast algorithm for computing the Discre

    Transform

    (Re)discovered by Cooley & Tukey in 196adopted thereafter

  • 8/6/2019 Fft Lecture

    5/90

    Fast Fourier Transform -

    erview

    Fast Fourier Transform - Overv

    J. W. Cooley and J. W. Tukey. An algorithmcalculation of complex Fourier series. MathComputation, 19:297301, 1965 A fast algorithm for computing the Discre

    Transform

    (Re)discovered by Cooley & Tukey in 196adopted thereafter

    Has a long and fascinating history

  • 8/6/2019 Fft Lecture

    6/90

    Fast Fourier Transform -

    erview

    urier Analysis

    Fourier Series

    Continuous Fourier Transform

    Discrete Fourier Transform

    Useful properties 6

    Applications Fourier Analysis

  • 8/6/2019 Fft Lecture

    7/90

    Fast Fourier Transform -

    erview

    urier Analysis

    Fourier Series

    Continuous Fourier Transform

    Discrete Fourier Transform

    Useful properties 6

    Applications

    Fourier Series

    Expresses a (real) periodic function x(t)trigonometric series (L < t < L)

    x(t) =1

    2a0 +

    n=1

    (an cosn

    Lt + b

    Coefficients can be computed by

    an =1

    L

    LL

    x(t) cosn

    Lt

    bn =1

    L

    LL

    x(t) sinn

    Lt

  • 8/6/2019 Fft Lecture

    8/90

    Fast Fourier Transform -

    erview

    urier Analysis

    Fourier Series

    Continuous Fourier Transform

    Discrete Fourier Transform

    Useful properties 6

    Applications

    Fourier Series

    Generalized to complex-valued functions

    x(t) =

    n=

    cnei nL

    t

    cn =1

    2L

    L

    L

    x(t)ein

    L

  • 8/6/2019 Fft Lecture

    9/90

    Fast Fourier Transform -

    erview

    urier Analysis

    Fourier Series

    Continuous Fourier Transform

    Discrete Fourier Transform

    Useful properties 6

    Applications

    Fourier Series

    Generalized to complex-valued functions

    x(t) =

    n=

    cnei nL

    t

    cn =1

    2L

    L

    L

    x(t)ein

    L

    Studied by D.Bernoulli and L.Euler

    Used by Fourier to solve the heat equatio

  • 8/6/2019 Fft Lecture

    10/90

    Fast Fourier Transform -

    erview

    urier Analysis

    Fourier Series

    Continuous Fourier Transform

    Discrete Fourier Transform

    Useful properties 6

    Applications

    Fourier Series

    Generalized to complex-valued functions

    x(t) =

    n=

    cnei nL

    t

    cn =1

    2L

    L

    L

    x(t)ein

    L

    Studied by D.Bernoulli and L.Euler

    Used by Fourier to solve the heat equatio

    Converges for almost all nice functions

    L2 etc.)

  • 8/6/2019 Fft Lecture

    11/90

    Fast Fourier Transform -

    erview

    urier Analysis

    Fourier Series

    Continuous Fourier Transform

    Discrete Fourier Transform

    Useful properties 6

    Applications

    Continuous Fourier Transform

    Generalization of Fourier Series for infinit

    x(t) =

    F(f)e2i f

    F(f) =

    x(t)e2i f t dt

    Can represent continuous, aperiodic sign

    Continuous frequency spectrum

  • 8/6/2019 Fft Lecture

    12/90

    Fast Fourier Transform -

    erview

    urier Analysis

    Fourier Series

    Continuous Fourier Transform

    Discrete Fourier Transform

    Useful properties 6

    Applications

    Discrete Fourier Transform

    If the signal X(k) is periodic, band-limitedNyquist frequency or higher, the DFT rep

    exactly14

    A(r) =N1

    k=0

    X(k)WrkN

    where WN = e

    2i

    N and r = 0 , 1 , . . . , N

    The inverse transform:

    X(j) =1

    N

    N1

    k=0

    A(k)W

    N

  • 8/6/2019 Fft Lecture

    13/90

    Fast Fourier Transform -

    erview

    urier Analysis

    Fourier Series

    Continuous Fourier Transform

    Discrete Fourier Transform

    Useful properties 6

    Applications

    Useful properties 6

    Orthogonality of basis functions

    N1

    k=0

    Wrk

    N WrkN = NN(r

    Linearity

    (Cyclic) Convolution theorem

    (x y)n =N1

    k=0

    x(k)y(

    F {(x y)} = F {X}F {Y

    Symmetries: real hermitian symmetrichermitian antisymmetric

    Shifting theorems

  • 8/6/2019 Fft Lecture

    14/90

    Fast Fourier Transform -

    erview

    urier Analysis

    Fourier Series

    Continuous Fourier Transform

    Discrete Fourier Transform

    Useful properties 6

    Applications

    Applications

    Approximation by trigonometric polynomi Data compression (MP3...)

    Spectral analysis of signals

    Frequency response of systems

    Partial Differential Equations

    Convolution via frequency domain Cross-correlation Multiplication of large integers Symbolic multiplication of polynomials

  • 8/6/2019 Fft Lecture

    15/90

    Fast Fourier Transform -

    erview

    ethods known by 1965

    Available methods

    Goertzels algorithm 7

    Methods known by

  • 8/6/2019 Fft Lecture

    16/90

    Fast Fourier Transform -

    erview

    ethods known by 1965

    Available methods

    Goertzels algorithm 7

    Available methods

    In many applications, large digitized dataavailable, but cannot be processed due torunning time of DFT

    All (publicly known) methods for efficient

    the symmetry of trigonometric functions,

    Goertzels method7

    is fastest known

  • 8/6/2019 Fft Lecture

    17/90

    Fast Fourier Transform -

    erview

    ethods known by 1965

    Available methods

    Goertzels algorithm 7

    Goertzels algorithm7

    Given a particular 0 < j < N 1, we wan

    A(j) =N1

    k=0

    x(k) cos

    2j

    Nk

    B(j) =

    N

    k=

  • 8/6/2019 Fft Lecture

    18/90

    Fast Fourier Transform -

    erview

    ethods known by 1965

    Available methods

    Goertzels algorithm 7

    Goertzels algorithm7

    Given a particular 0 < j < N 1, we wan

    A(j) =N1

    k=0

    x(k) cos

    2j

    Nk

    B(j) =

    N

    k=

    Recursively compute the sequence {u(s)

    u(s) = x(s) + 2cos

    2j

    N

    u(s +

    u(N) = u(N+ 1) = 0

  • 8/6/2019 Fft Lecture

    19/90

    Fast Fourier Transform -

    erview

    ethods known by 1965

    Available methods

    Goertzels algorithm 7

    Goertzels algorithm7

    Given a particular 0 < j < N 1, we wan

    A(j) =N1

    k=0

    x(k) cos

    2j

    Nk

    B(j) =

    N

    k=

    Recursively compute the sequence {u(s)

    u(s) = x(s) + 2cos

    2j

    N

    u(s +

    u(N) = u(N+ 1) = 0

    Then

    B(j) = u(1) sin

    2j

    N

    A(j) = x(0) + cos

    2j

    N

    u(1

  • 8/6/2019 Fft Lecture

    20/90

    Fast Fourier Transform -

    erview

    ethods known by 1965

    Available methods

    Goertzels algorithm 7

    Goertzels algorithm7

    Requires N multiplications and only one

    Roundoff errors grow rapidly5

    Excellent for computing a very small num(< log N)

    Can be implemented using only 2 interme

    locations and processing input signal as Used today for Dual-Tone Multi-Frequenc

    dialing)

  • 8/6/2019 Fft Lecture

    21/90

    Fast Fourier Transform -

    erview

    spiration for the FFT

    Factorial experiments

    Goods paper 9

    Goods results - summary

    Inspiration for the

  • 8/6/2019 Fft Lecture

    22/90

    Fast Fourier Transform -

    erview

    spiration for the FFT

    Factorial experiments

    Goods paper 9

    Goods results - summary

    Factorial experiments

    Purpose: investigate the effect of n factors quantity Each factor can have t distinct levels. Fo

    t = 2, so there are 2 levels: + and .

    So we conduct tn experiments (for every combination of the n factors) and calcula Main effect of a factor: what is the ave

    output as the factor goes from to Interaction effects between two or mo

    the difference in output between the cfactors are present compared to whenis present?

  • 8/6/2019 Fft Lecture

    23/90

    Fast Fourier Transform -

    erview

    spiration for the FFT

    Factorial experiments

    Goods paper 9

    Goods results - summary

    Factorial experiments

    Example of t = 2, n = 3 experiment Say we have factors A, B, C which can be

    example, they can represent levels of 3 dto patients

    Perform 23 measurements of some quantpressure

    Investigate how each one of the drugs anaffect the blood pressure

  • 8/6/2019 Fft Lecture

    24/90

    Fast Fourier Transform -

    erview

    spiration for the FFT

    Factorial experiments

    Goods paper 9

    Goods results - summary

    Factorial experiments

    Exp. Combination A B C Bl

    1 (1)

    2 a +

    3 b + 4 ab + +

    5 c +

    6 ac + +

    7 bc + +

    8 abc + + +

  • 8/6/2019 Fft Lecture

    25/90

    Fast Fourier Transform -

    erview

    spiration for the FFT

    Factorial experiments

    Goods paper 9

    Goods results - summary

    Factorial experiments

    The main effect of A is(ab b) + (a (1)) + (abc bc) + (ac

    The interaction of A, B is(ab b) (a (1)) ((abc bc) (ac

    etc.

  • 8/6/2019 Fft Lecture

    26/90

    Fast Fourier Transform -

    erview

    spiration for the FFT

    Factorial experiments

    Goods paper 9

    Goods results - summary

    Factorial experiments

    The main effect of A is(ab b) + (a (1)) + (abc bc) + (ac

    The interaction of A, B is(ab b) (a (1)) ((abc bc) (ac

    etc.

    If the main effects and interactions are arrathen we can write y = Ax where

    A =

    1 1 1 1 1 1

    1 1 1 1 1 1

    1 1 1 1 1 1

    1 1 1 1 1 11 1 1 1 1 1

    1 1 1 1 1 1

    1 1 1 1 1 1

    1 1 1 1 1 1

  • 8/6/2019 Fft Lecture

    27/90

    Fast Fourier Transform -

    erview

    spiration for the FFT

    Factorial experiments

    Goods paper 9

    Goods results - summary

    Factorial experiments

    F.Yates devised15 an iterative n-step algosimplifies the calculation. Equivalent to dA = Bn, where (for our case of n = 3):

    B =

    1 1 0 0 0 0

    0 0 1 1 0 0

    0 0 0 0 1 1

    0 0 0 0 0 0

    1 1 0 0 0 0

    0 0 1 1 0 0

    0 0 0 0 1

    0 0 0 0 0 0

    so that much less operations are required

  • 8/6/2019 Fft Lecture

    28/90

    Fast Fourier Transform -

    erview

    spiration for the FFT

    Factorial experiments

    Goods paper 9

    Goods results - summary

    Goods paper9

    I. J. Good. The interaction algorithm and pranalysis. Journal Roy. Stat. Soc., 20:3613 Essentially develops a prime-factor FFT

    The idea is inspired by Yates efficient algproves general theorems which can be aefficiently multiplying a N-vector by certa

    The work remains unnoticed until Cooley

    Good meets Tukey in 1956 and tells him method

    It is revealed later4 that the FFT had not much earlier by a coincedence

  • 8/6/2019 Fft Lecture

    29/90

    Fast Fourier Transform -

    erview

    spiration for the FFT

    Factorial experiments

    Goods paper 9

    Goods results - summary

    Goods paper9

    Definition 1 Let M(i) be t t matrices, i =direct product

    M(1) M(2) M(n

    is atn tn matrix C whose elements are

    Cr,s =n

    i=1

    M(i)ri,si

    wherer = (r1, . . . , rn) ands = (s1, . . . sn) arepresentation of the index

  • 8/6/2019 Fft Lecture

    30/90

    Fast Fourier Transform -

    erview

    spiration for the FFT

    Factorial experiments

    Goods paper 9

    Goods results - summary

    Goods paper9

    Definition 1 Let M(i) be t t matrices, i =direct product

    M(1) M(2) M(n

    is atn tn matrix C whose elements are

    Cr,s =n

    i=1

    M(i)ri,si

    wherer = (r1, . . . , rn) ands = (s1, . . . sn) arepresentation of the index

    Theorem 1 For every M, M[n] = An where

    Ar,s = {Mr1,sns1r2

    s2r3 . . .

    snrn

    Interpretation: A has at most tn+1 nonzero

    a given B we can find M such that M[n] = Bcompute Bx = Anx in O(ntn) instead of O(

  • 8/6/2019 Fft Lecture

    31/90

    Fast Fourier Transform -

    erview

    spiration for the FFT

    Factorial experiments

    Goods paper 9

    Goods results - summary

    Goods paper9

    Goods observations For 2n factorial experiment:

    M =

    1 1

    1 1

  • 8/6/2019 Fft Lecture

    32/90

    Fast Fourier Transform -

    erview

    spiration for the FFT

    Factorial experiments

    Goods paper 9

    Goods results - summary

    Goods paper9

    Goods observations n-dimensional DFT, size N in each dimen

    A(s1, . . . , sn) =(N1,...,N1)

    (r1,...,rn)

    x(r)Wr1sN

    can be represented as

    A = [n]x, = {WrsN} r, s = 0

    By the theorem, A = nx where is spa

  • 8/6/2019 Fft Lecture

    33/90

    Fast Fourier Transform -

    erview

    spiration for the FFT

    Factorial experiments

    Goods paper 9

    Goods results - summary

    Goods paper9

    Goods observations n-dimensional DFT, size N in each dimen

    A(s1, . . . , sn) =(N1,...,N1)

    (r1,...,rn)

    x(r)Wr1sN

    can be represented as

    A = [n]x, = {WrsN} r, s = 0

    By the theorem, A = nx where is spa

    n-dimensional DFT, Ni in each dimension

    A = ((1) (n))x = (1)

    (i) = {WrsNi}

    (i) = IN1 (i) INn

  • 8/6/2019 Fft Lecture

    34/90

    Fast Fourier Transform -

    erview

    spiration for the FFT

    Factorial experiments

    Goods paper 9

    Goods results - summary

    Goods paper9

    If N = N1N2 and (N1, N2) = 1 then N-poinrepresented as 2-D DFT

    r = N2r1 + N1r2

    (Chinese Remainder Thm.)s = N2s1(N12 ) mod N

    s s1 mod N1, s

    These are index mappings s (s1, s2) and

    A(s) = A(s1, s2) =N

    r=0

    x(r)WrsN =N11

    r1=0

    (substitute r, s from above) =N11

    r1=0

    N21

    r2=0

    x(r1, r2)W

    This is exactly 2-D DFT seen above!

  • 8/6/2019 Fft Lecture

    35/90

    Fast Fourier Transform -

    erview

    spiration for the FFT

    Factorial experiments

    Goods paper 9

    Goods results - summary

    Goods paper9

    So, as above, we can write

    A = x

    = P((1) (2))Q1

    = P(1)(2)Q1x

    where P and Q are permutation matrices. TN(N1 + N2) multiplications instead of N

    2.

  • 8/6/2019 Fft Lecture

    36/90

    Fast Fourier Transform -

    erview

    spiration for the FFT

    Factorial experiments

    Goods paper 9

    Goods results - summary

    Goods results - summary

    n-dimensionanal, size Ni in each dimens

    n

    Ni

    n

    Ni

    instead of

    1-D, size N = n

    Ni

    where Nis are mutu

    n

    Ni

    N instead of N

    Still thinks that other tricks such as usinand sines and cosines of known angles museful

  • 8/6/2019 Fft Lecture

    37/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    The article

  • 8/6/2019 Fft Lecture

    38/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    James W.Cooley (1926-)

    1949 - B.A. in Mathematics(Manhattan College, NY)

    1951 - M.A. in Mathematics(Columbia University)

    1961 - Ph.D in Applied Math-

    ematics (Columbia Univer-sity)

    1962 - Joins IBM Watson Research Cent

    The FFT is considered his most significanmathematics

    Member of IEEE Digital Signal Processin

    Awarded IEEE fellowship on his works on

    Contributed much to establishing the term

  • 8/6/2019 Fft Lecture

    39/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    John Wilder Tukey (1915-2000)

    1936 - Bachelor in Chem-istry (Brown University)

    1937 - Master in Chemistry(Brown University)

    1939 - PhD in Mathemat-

    ics (Princeton) for Denu-merability in Topology

    During WWII joins Fire Control Researchcontributes to war effort (mainly invlolving

    From 1945 also works with Shannon andBell Labs

    Besides co-authoring the FFT in 1965, hacontributions to statistics

  • 8/6/2019 Fft Lecture

    40/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    The story - meeting

    Richard Garwin (Columbia University IBMcenter) meets John Tukey (professor of mPrinceton) at Kennedys Scientific Adviso1963

  • 8/6/2019 Fft Lecture

    41/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    The story - meeting

    Richard Garwin (Columbia University IBMcenter) meets John Tukey (professor of mPrinceton) at Kennedys Scientific Adviso1963 Tukey shows how a Fourier series of l

    N = ab is composite can be expresse

    of b-point subseries, thereby reducingcomputations from N2 to N(a + b)

  • 8/6/2019 Fft Lecture

    42/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    The story - meeting

    Richard Garwin (Columbia University IBMcenter) meets John Tukey (professor of mPrinceton) at Kennedys Scientific Adviso1963 Tukey shows how a Fourier series of l

    N = ab is composite can be expresse

    of b-point subseries, thereby reducingcomputations from N2 to N(a + b)

    Notes that if N is a power of two, the arepeated, giving Nlog2 N operations

  • 8/6/2019 Fft Lecture

    43/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    The story - meeting

    Richard Garwin (Columbia University IBMcenter) meets John Tukey (professor of mPrinceton) at Kennedys Scientific Adviso1963 Tukey shows how a Fourier series of l

    N = ab is composite can be expresse

    of b-point subseries, thereby reducingcomputations from N2 to N(a + b)

    Notes that if N is a power of two, the arepeated, giving Nlog2 N operations

    Garwin, being aware of applications inwould greatly benefit from an efficient

    realizes the importance of this and fropushes the FFT at every opportunity

  • 8/6/2019 Fft Lecture

    44/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    The story - publication

    Garwin goes to Jim Cooley (IBM Watsonwith the notes from conversation with Tukimplement the algorithm

  • 8/6/2019 Fft Lecture

    45/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    The story - publication

    Garwin goes to Jim Cooley (IBM Watsonwith the notes from conversation with Tukimplement the algorithm

    Says that he needs the FFT for computin

    transform of spin orientations of He3. It iswhat Garwin really wanted was to be ableexposions from seismic data

  • 8/6/2019 Fft Lecture

    46/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    The story - publication

    Garwin goes to Jim Cooley (IBM Watsonwith the notes from conversation with Tukimplement the algorithm

    Says that he needs the FFT for computin

    transform of spin orientations of He3. It iswhat Garwin really wanted was to be ableexposions from seismic data

    Cooley finally writes a 3-D in-place FFT aGarwin

  • 8/6/2019 Fft Lecture

    47/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    The story - publication

    Garwin goes to Jim Cooley (IBM Watsonwith the notes from conversation with Tukimplement the algorithm

    Says that he needs the FFT for computin

    transform of spin orientations of He3. It iswhat Garwin really wanted was to be ableexposions from seismic data

    Cooley finally writes a 3-D in-place FFT aGarwin

    The FFT is exposed to a number of math

  • 8/6/2019 Fft Lecture

    48/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    The story - publication

    Garwin goes to Jim Cooley (IBM Watsonwith the notes from conversation with Tukimplement the algorithm

    Says that he needs the FFT for computin

    transform of spin orientations of He3. It iswhat Garwin really wanted was to be ableexposions from seismic data

    Cooley finally writes a 3-D in-place FFT aGarwin

    The FFT is exposed to a number of math

    It is decided that the algorithm should be

    domain to prevent patenting

  • 8/6/2019 Fft Lecture

    49/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    The story - publication

    Garwin goes to Jim Cooley (IBM Watsonwith the notes from conversation with Tukimplement the algorithm

    Says that he needs the FFT for computin

    transform of spin orientations of He3. It iswhat Garwin really wanted was to be ableexposions from seismic data

    Cooley finally writes a 3-D in-place FFT aGarwin

    The FFT is exposed to a number of math

    It is decided that the algorithm should be

    domain to prevent patenting Cooley writes the draft, Tukey adds some

    Goods article and the paper is submittedComputation in August 1964

  • 8/6/2019 Fft Lecture

    50/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    The story - publication

    Garwin goes to Jim Cooley (IBM Watsonwith the notes from conversation with Tukimplement the algorithm

    Says that he needs the FFT for computin

    transform of spin orientations of He3. It iswhat Garwin really wanted was to be ableexposions from seismic data

    Cooley finally writes a 3-D in-place FFT aGarwin

    The FFT is exposed to a number of math

    It is decided that the algorithm should be

    domain to prevent patenting Cooley writes the draft, Tukey adds some

    Goods article and the paper is submittedComputation in August 1964

    Published in April 1965

  • 8/6/2019 Fft Lecture

    51/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    What if...

    Good reveals in 19934 that Garwin visite

    Garwin told me with great enthusiasm aexperimental work... Unfortunately, not wthe subject..., I didnt mention my FFT woalthough I had intended to do so. He wro

    9, 1976 and said: Had we talked about ihave stolen [publicized] it from you insteaTukey...... That would have completely cof the FFT

  • 8/6/2019 Fft Lecture

    52/90

    Cooley-Tukey FFT

    A(j) = N1k=0 X(k)WjkN N = N1N2

  • 8/6/2019 Fft Lecture

    53/90

    Cooley-Tukey FFT

    A(j) = N1k=0 X(k)WjkN N = N1N2

    Convert the 1-D indices into 2-D

    j = j0 + j1N1 j0 = 0 , . . . , N1 1 j1 =

    k = k0 + k1N2 k0 = 0 , . . . , N2 1 k1 =

  • 8/6/2019 Fft Lecture

    54/90

    Cooley-Tukey FFT

    A(j) = N1k=0 X(k)WjkN N = N1N2

    Convert the 1-D indices into 2-D

    j = j0 + j1N1 j0 = 0 , . . . , N1 1 j1 =

    k = k0 + k1N2 k0 = 0 , . . . , N2 1 k1 =

    Decimate the original sequence into N2 N1-subsequ

  • 8/6/2019 Fft Lecture

    55/90

    Cooley-Tukey FFT

    A(j) = N1k=0 X(k)WjkN N = N1N2

    Convert the 1-D indices into 2-D

    j = j0 + j1N1 j0 = 0 , . . . , N1 1 j1 =

    k = k0 + k1N2 k0 = 0 , . . . , N2 1 k1 =

    Decimate the original sequence into N2 N1-subsequ

    A(j1, j0) =N1

    k=0

    X(k)WjkN

  • 8/6/2019 Fft Lecture

    56/90

    Cooley-Tukey FFT

    A(j) = N1k=0 X(k)WjkN N = N1N2

    Convert the 1-D indices into 2-D

    j = j0 + j1N1 j0 = 0 , . . . , N1 1 j1 =

    k = k0 + k1N2 k0 = 0 , . . . , N2 1 k1 =

    Decimate the original sequence into N2 N1-subsequ

    A(j1, j0) =N1

    k=0

    X(k)WjkN

    =

    N21

    k0=0

    N11

    k1=0X(k1, k0)W

    (j0+j1N1)(k0+k1N1N2

  • 8/6/2019 Fft Lecture

    57/90

    Cooley-Tukey FFT

    A(j) = N1k=0 X(k)WjkN N = N1N2

    Convert the 1-D indices into 2-D

    j = j0 + j1N1 j0 = 0 , . . . , N1 1 j1 =

    k = k0 + k1N2 k0 = 0 , . . . , N2 1 k1 =

    Decimate the original sequence into N2 N1-subsequ

    A(j1, j0) =N1

    k=0

    X(k)WjkN

    =

    N21

    k0=0

    N11

    k1=0X(k1, k0)W

    (j0+j1N1)(k0+k1N1N2

    =N21

    k0=0

    Wj0k0N W

    j1k0N2

    N11

    k1=0

    X(k1, k0)Wj0kN1

  • 8/6/2019 Fft Lecture

    58/90

    Cooley-Tukey FFT

    A(j) = N1k=0 X(k)WjkN N = N1N2

    Convert the 1-D indices into 2-D

    j = j0 + j1N1 j0 = 0 , . . . , N1 1 j1 =

    k = k0 + k1N2 k0 = 0 , . . . , N2 1 k1 =

    Decimate the original sequence into N2 N1-subsequ

    A(j1, j0) =N1

    k=0

    X(k)WjkN

    =

    N21

    k0=0

    N11

    k1=0X(k1, k0)W

    (j0+j1N1)(k0+k1N2)N1N2

    =N21

    k0=0

    Wj0k0N W

    j1k0N2

    N11

    k1=0

    X(k1, k0)Wj0k1N1

    =

  • 8/6/2019 Fft Lecture

    59/90

    Cooley-Tukey FFT

    The sequences Xj0 have N elements and require NNcompute

    Given these, the array A requires NN2 operations to

    Overall N(N1 + N2)

    In general, if N = ni=0 Ni then N(ni=0 Ni) operation

  • 8/6/2019 Fft Lecture

    60/90

    Cooley-Tukey FFT

    If N1 = N2 = = Nn = 2 (i.e. N= 2n):

    j = jn12n1 + +j0, k = kn12

    A(jn1, . . . ,j0) =1

    k0=0

    1

    kn1=0

    X(kn1, . . . , k0)WjN

    = k0

    Wjk0N

    k1

    Wjk12N

    kn1

    Wjkn1N

    Xj0 (kn2, . . . , k0) = kn1

    X(kn1, . . . , k0)Wj0kn12

    Xj1,j0 (kn3, . . . , k0) = kn2

    Wj0kn24

    Xj0 (kn2, . . . , k0)Wj1k2

    Xjm1,...,j0 (knm1, . . . , k0) = knm

    W(j0++jm22m2)knm2m

    Xjm2,

    Wjm1knm2

  • 8/6/2019 Fft Lecture

    61/90

    Cooley-Tukey FFT

    Finally, A(jn1, . . . , j0) = Xjn1,...,j0 Two important properties:

    In-place: only two terms are involved in the calculaintermediate pair Xjm1,...,j0(. . . ) jm1 = 0, 1: Xjm2,...,j0(0 , . . . )

    Xjm2,...,j0(1 , . . . )So the values can be overwritten and no more addused

    Bit-reversal It is convenient to store Xjm1,...,j0(knm1, . . . , k0

    j02n1 + + jm12

    nm + knm12mn1 +

    The result must be reversed Cooley & Tukey didnt realize that the algorithm in fac

    DFTs from smaller ones with some multiplications alo

    The terms W(j0++jm22

    m2)knm2m are later called

    6 tw

  • 8/6/2019 Fft Lecture

    62/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    FFT properties

    Roundoff error significantly reduced com

    formula6

    A lower bound of 12

    n log2 n operations ov

    algorithms is proved12, so FFT is in a sen

    Two canonical FFTs

    Decimation-in-Time: the Cooley & TukEquivalent to taking N1 = N/2, N2are the DFT coefficients of even-numbXj0(1) - those of odd-numbered. The fi

    simply linear combination of the two. Decimation-in-Frequency (Tukey-Sand

    N1 = 2, N2 = N/2. Then Y() = Xeven-numbered and odd-numbered saZ() = X1() is the difference. The evcoefficients are the DFT of Y, and theweighted DFT of Z.

  • 8/6/2019 Fft Lecture

    63/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    DIT signal flow

  • 8/6/2019 Fft Lecture

    64/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    DIT signal flow

  • 8/6/2019 Fft Lecture

    65/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    DIT signal flow

  • 8/6/2019 Fft Lecture

    66/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    DIF signal flow

  • 8/6/2019 Fft Lecture

    67/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    DIF signal flow

  • 8/6/2019 Fft Lecture

    68/90

    Fast Fourier Transform -

    erview

    e article

    ames W.Cooley (1926-)

    ohn Wilder Tukey

    915-2000)

    The story - meeting

    The story - publication

    What if...

    Cooley-Tukey FFT

    FFT propertiesDIT signal flow

    DIF signal flow

    DIF signal flow

  • 8/6/2019 Fft Lecture

    69/90

    Fast Fourier Transform -

    erview

    wly discovered history of FFT

    What happened after the

    blication

    Danielson-Lanczos

    Gauss Newly discovered histo

  • 8/6/2019 Fft Lecture

    70/90

    Fast Fourier Transform -

    erview

    wly discovered history of FFT

    What happened after the

    blication

    Danielson-Lanczos

    Gauss

    What happened after the publi

    Rudnicks note13 mentions Danielson-La

    19423. Although ... less elegant and is pterms of real quantities, it yields the samebinary form of the Cooley-Tukey algorithmnumber of arithmetical operations.

    Historical Notes on FFT2

    Also mentions Danielson-Lanczos Thomas and Prime Factor algorithm (

    distinct from Cooley-Tukey FFT

    A later investigation10 revealed that Gausdiscovered the Cooley-Tukey FFT in 1805

  • 8/6/2019 Fft Lecture

    71/90

    Danielson-Lanczos

    G. C. Danielson and C. Lanczos. Some improvementFourier analysis and their application to X-ray scatterFranklin Institute, 233:365380 and 435452, 1942

  • 8/6/2019 Fft Lecture

    72/90

    Danielson-Lanczos

    G. C. Danielson and C. Lanczos. Some improvementFourier analysis and their application to X-ray scatterFranklin Institute, 233:365380 and 435452, 1942

    ... the available standard forms become impractical fof coefficients. We shall show that, by a certain transfit is possible to double the number of ordinates with othan double the labor

  • 8/6/2019 Fft Lecture

    73/90

    Danielson-Lanczos

    G. C. Danielson and C. Lanczos. Some improvementFourier analysis and their application to X-ray scatterFranklin Institute, 233:365380 and 435452, 1942

    ... the available standard forms become impractical fof coefficients. We shall show that, by a certain transfit is possible to double the number of ordinates with othan double the labor

    Concerned with improving efficiency of hand calculat

  • 8/6/2019 Fft Lecture

    74/90

    Danielson-Lanczos

    G. C. Danielson and C. Lanczos. Some improvementFourier analysis and their application to X-ray scatterFranklin Institute, 233:365380 and 435452, 1942

    ... the available standard forms become impractical fof coefficients. We shall show that, by a certain transfit is possible to double the number of ordinates with othan double the labor

    Concerned with improving efficiency of hand calculat

    We shall now describe a method which eliminates thcomplicated schemes by reducing ... analysis for 4n canalyses for 2n coefficients

  • 8/6/2019 Fft Lecture

    75/90

    Danielson-Lanczos

    G. C. Danielson and C. Lanczos. Some improvementFourier analysis and their application to X-ray scatterFranklin Institute, 233:365380 and 435452, 1942

    ... the available standard forms become impractical fof coefficients. We shall show that, by a certain transfit is possible to double the number of ordinates with othan double the labor

    Concerned with improving efficiency of hand calculat

    We shall now describe a method which eliminates thcomplicated schemes by reducing ... analysis for 4n canalyses for 2n coefficients

    If desired, this reduction process can be applied twic(???!!)

    Adoping these improvements the approximate timesfor 8 coefficients, 25 min. for 16, 60 min. for 32 and 1( 0.37Nlog2 N)

  • 8/6/2019 Fft Lecture

    76/90

    Danielson-Lanczos

    Take the DIT Cooley-Tukey with N1 = N/2, N2 = 2

    A(j0 +N

    2j1) =

    N/21

    k1=0

    X(2k1)Wj0k1N/2 + W

    j0+N

    2j1

    N

    N/21

    k1=0

  • 8/6/2019 Fft Lecture

    77/90

    Danielson-Lanczos

    Take the DIT Cooley-Tukey with N1 = N/2, N2 = 2

    A(j0 +N

    2j1) =

    N/21

    k1=0

    X(2k1)Wj0k1N/2 + W

    j0+N

    2j1

    N

    N/21

    k1=0

    Danielson-Lanczos showed completely analogous re

    framework of real trigonometric series: ...the contribution of the even ordinates to a Four

    coefficients is exactly the same as the contributionordinates to a Fourier analysis of 2n coefficients

    ..contribution of the odd ordinates is also reducibscheme... by means of a transformation process i

    phase difference ...we see that, apart from the weight factors 2cos

    2sin(/4n)k, the calculation is identical to the coshalf the number of ordinates

  • 8/6/2019 Fft Lecture

    78/90

    Danielson-Lanczos

    Take the DIT Cooley-Tukey with N1 = N/2, N2 = 2

    A(j0 +N

    2j1) =

    N/21

    k1=0

    X(2k1)Wj0k1N/2 + W

    j0+N

    2j1

    N

    N/21

    k1=0

    Danielson-Lanczos showed completely analogous re

    framework of real trigonometric series: ...the contribution of the even ordinates to a Four

    coefficients is exactly the same as the contributionordinates to a Fourier analysis of 2n coefficients

    ..contribution of the odd ordinates is also reducibscheme... by means of a transformation process i

    phase difference ...we see that, apart from the weight factors 2cos

    2sin(/4n)k, the calculation is identical to the coshalf the number of ordinates

    These weight factors are exactly the twiddle fac

  • 8/6/2019 Fft Lecture

    79/90

    Fast Fourier Transform -

    erview

    wly discovered history of FFT

    What happened after the

    blication

    Danielson-Lanczos

    Gauss

    Gauss

    Herman H. Goldstine writes in a footnoteNumerical Analysis from the 16th through

    (1977)8:This fascinating work of Gauss was nrediscovered by Cooley and Tukey in paper in 1965

    This goes largely unnoticed until the rese

    Johnson and Burrus10 in 1985 Essentially develops an DIF FFT for tw

    sequences. Gives examples for N = Declares that it can be generalized to

    factors, although no examples are giv Uses the algorithm for solving the proorbit parameters of asteroids

    The treatise was published posthumothen other numerical methods were prnobody found this interesting enough.

  • 8/6/2019 Fft Lecture

    80/90

    Fast Fourier Transform -

    erview

    pact

    mpact

    Further developments

    Concluding thoughts

    Impact

  • 8/6/2019 Fft Lecture

    81/90

    Fast Fourier Transform -

    erview

    pact

    mpact

    Further developments

    Concluding thoughts

    Impact

    Certainly a classic

    Two special issues of IEEE trans. on Aud

    Electroacoustics devoted entirely to FFT

    Arden House Workshop on FFT11

    Brought together people of very divers

    someday radio tuners will operate witunits. I have heard this suggested witbut one can speculate. - J.W.Cooley

    Many people discovered the DFT via th

  • 8/6/2019 Fft Lecture

    82/90

    Fast Fourier Transform -

    erview

    pact

    mpact

    Further developments

    Concluding thoughts

    Further developments

    Real-FFT (DCT...)

    Parallel FFT

    FFT for prime N

    ...

  • 8/6/2019 Fft Lecture

    83/90

    Fast Fourier Transform -

    erview

    pact

    mpact

    Further developments

    Concluding thoughts

    Concluding thoughts

    It is obvious that prompt recognition and significant achievments is an important g

  • 8/6/2019 Fft Lecture

    84/90

    Fast Fourier Transform -

    erview

    pact

    mpact

    Further developments

    Concluding thoughts

    Concluding thoughts

    It is obvious that prompt recognition and significant achievments is an important g

    However, the publication in itself may not

  • 8/6/2019 Fft Lecture

    85/90

    Fast Fourier Transform -

    erview

    pact

    mpact

    Further developments

    Concluding thoughts

    Concluding thoughts

    It is obvious that prompt recognition and significant achievments is an important g

    However, the publication in itself may not

    Communication between mathematiciansanalysts and workers in a very wide rang

    can be very fruitful

  • 8/6/2019 Fft Lecture

    86/90

    Fast Fourier Transform -

    erview

    pact

    mpact

    Further developments

    Concluding thoughts

    Concluding thoughts

    It is obvious that prompt recognition and significant achievments is an important g

    However, the publication in itself may not

    Communication between mathematiciansanalysts and workers in a very wide rang

    can be very fruitful Always seek new analytical methods and

    increase in processing speed

  • 8/6/2019 Fft Lecture

    87/90

    Fast Fourier Transform -

    erview

    pact

    mpact

    Further developments

    Concluding thoughts

    Concluding thoughts

    It is obvious that prompt recognition and significant achievments is an important g

    However, the publication in itself may not

    Communication between mathematiciansanalysts and workers in a very wide rang

    can be very fruitful Always seek new analytical methods and

    increase in processing speed

    Careful attention to a review of old literaturewards

  • 8/6/2019 Fft Lecture

    88/90

    Fast Fourier Transform -

    erview

    pact

    mpact

    Further developments

    Concluding thoughts

    Concluding thoughts

    It is obvious that prompt recognition and significant achievments is an important g

    However, the publication in itself may not

    Communication between mathematiciansanalysts and workers in a very wide rang

    can be very fruitful Always seek new analytical methods and

    increase in processing speed

    Careful attention to a review of old literaturewards

    Do not publish papers in Journal of Frank

  • 8/6/2019 Fft Lecture

    89/90

    Fast Fourier Transform -

    erview

    pact

    mpact

    Further developments

    Concluding thoughts

    Concluding thoughts

    It is obvious that prompt recognition and significant achievments is an important g

    However, the publication in itself may not

    Communication between mathematiciansanalysts and workers in a very wide rang

    can be very fruitful Always seek new analytical methods and

    increase in processing speed

    Careful attention to a review of old literaturewards

    Do not publish papers in Journal of Frank

    Do not publish papers in neo-classic Lati

  • 8/6/2019 Fft Lecture

    90/90

    References

    1. J. W. Cooley and J. W. Tukey. An algorithm for the machine calculation of complex Fourier series.Mathematics of Computation, 19:297301, 1965.

    2. J. W. Cooley, P. A. Lewis, and P. D. Welch. History of the fast Fourier transform. In Proc. IEEE,volume 55, pages 16751677, October 1967.

    3. G. C. Danielson and C. Lanczos. Some improvements in practical Fourier analysis and their appli-cation to X-ray scattering from liquids. J. Franklin Institute, 233:365380 and 435452, 1942.

    4. D.L.Banks. A conversation with I. J. Good. Statistical Science, 11(1):119, 1996.

    5. W. M. Gentleman. An error analysis of Goertzels (Watts) method for computing Fourier coeffi-cients. Comput. J., 12:160165, 1969.

    6. W. M. Gentleman and G. Sande. Fast Fourier transformsfor fun and profit. In Fall Joint ComputerConference, volume 29 ofAFIPS Conference Proceedings, pages 563578. Spartan Books, Washington,D.C., 1966.

    7. G. Goertzel. An algorithm for the evaluation of finite trigonometric series. The American Mathe-matical Monthly, 65(1):3435, January 1958.

    8. Herman H. Goldstine. A History of Numerical Analysis from the 16th through the 19th Century.Springer-Verlag, New York, 1977. ISBN 0-387-90277-5.

    9. I. J. Good. The interaction algorithm and practical Fourier analysis. Journal Roy. Stat. Soc., 20:361372, 1958.

    10. M. T. Heideman, D. H. Johnson, and C. S. Burrus. Gauss and the history of the Fast FourierTransform. Archive for History of Exact Sciences, 34:265267, 1985.

    11. IEEE. Special issue on fast Fourier transform. IEEE Trans. on Audio and Electroacoustics, AU-17:65186, 1969.

    12. Morgenstern. Note on a lower bound of the linear complexity of the fast Fourier transform. JACM:Journal of the ACM, 20, 1973.

    13. Philip Rudnick. Note on the calculation of Fourier series (in Technical Notes and Short Papers).j-MATH-COMPUT, 20(95):429430, July 1966. ISSN 0025-5718.