![Page 1: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/1.jpg)
Faster Algorithms for Sparse Fourier Transform
Haitham HassaniehPiotr Indyk Dina Katabi Eric Price
MIT
Material from: •Hassanieh, Indyk, Katabi, Price, “Simple and Practical Algorithms for Sparse Fourier Transform, SODA’12.•Hassanieh, Indyk, Katabi, Price, “Nearly Optimal Sparse Fourier Transform”, STOC’12.
![Page 2: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/2.jpg)
The Discrete Fourier Transform• Discrete Fourier Transform:
– Given: a signal ∈ – Goal: compute the frequency vector such that for ∈ 1… :
∑ /
• Fundamental tool:– Compression (audio, image, video)– Signal processing– Data analysis– Wireless Communication– …
• FFT : O log timeSampled Audio Data (Time)DFT of Audio Samples (Frequency)
![Page 3: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/3.jpg)
Sparse Fourier Transform
• Often the Fourier transform is dominated by a small number of “peaks”
– Only few of the frequency coefficients are nonzero. – An exactly k‐sparse signal has only k nonzero frequency coefficients.– In practice : approximate a sparse signal using the k largest peaks.
• Problem : Can we recover the k‐sparse frequency spectrum faster than FFT?
Time Domain Signal Sparse Frequency Spectrum
Approximately Sparse Frequency Spectrum
![Page 4: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/4.jpg)
Previous Work
• Algorithms:– Boolean cube : [KM92], [GL89]. What about ?– Complex FT: [Mansour‐92, GGIMS02, AGS03, GMS05, Iwen10, Aka10]
• Best running time: [GMS05] O– In theory : Improves over FFT for / log3
– In Practice : Large constants; need / 40,000 to beat FFT
• Goal:– Theory: improve over FFT for all values of – Practice: faster runtime than FFT.
![Page 5: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/5.jpg)
Our results• Randomized algorithms, with constant probability of success
• Exactly ‐sparse case, recover : log– Optimal if FFT optimal
• Approximately ‐sparse case, recover ′ :– Let Err min
– l2/l2 guarantee ′ Err : log log /• Improves over FFT for any ≪
– l∞/l2 guarantee ′ Err : log log• Improves over FFT for ≪ /log
![Page 6: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/6.jpg)
Sparse FFT ‐ Algorithm
![Page 7: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/7.jpg)
Intuitionn‐point DFT : log
Frequency DomainTime Domain Signal
n‐point DFT of first B terms : log
Cut off Time signal Frequency Domain
Boxcar ∗sinc
B‐point DFT of first B terms: log
First B samples Frequency Domain Subsample ∗ sinc
Alias Boxcar
![Page 8: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/8.jpg)
• “Hashes” the Fourier coefficients into buckets in O( log ) time
• Issues– Leakage :
– Given these buckets, how can we estimate the locations and values the large frequencies?
Framework
n‐point DFTof all n samples
B‐point DFTof first B sample
n frequencies hash into B buckets
Subsample ∗ sincSubsample ∗ Filter
![Page 9: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/9.jpg)
Filter: Sinc
– Polynomial decay– Leaking many buckets
Boxcar Sinc
![Page 10: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/10.jpg)
– Exponential decay
– Leaking to buckets
Filter: Gaussian
Gaussian Gaussian
![Page 11: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/11.jpg)
Filters: Wider Gaussian
– Exponential decay– Leaking to <1 buckets– But trivial contribution to the correct bucket
Wider Gaussian Narrow Gaussian
![Page 12: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/12.jpg)
Filters: Sinc × Gaussian
– Boxcar size / : / frequencies hash into each bucket – Still exponential decay– Leaking to at most 1 bucket – Sufficient contribution to the correct bucket /2 , /2– Replace Gaussians with “Dolph‐Chebyshev window functions”
Sinc × Gaussian Boxcar ∗ Gaussian
![Page 13: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/13.jpg)
Finding the support• = B‐point DFT = Subsample
• Assume no collisions:– At most one large frequency hashes into each bucket.– Large frequency hashes to bucket :
= ∆ leakage
– Recall: DFT( ) = /
– = B‐point DFT := / ∆ leakage
![Page 14: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/14.jpg)
Finding the support• = B‐point DFT = Subsample
• Assume no collisions:– At most one large frequency hashes into each bucket.– Large frequency hashes to bucket :
= ∆= / ∆
∠ mod∆
– Find all frequencies in 2 log
![Page 15: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/15.jpg)
Random Hashing
• Some Large frequencies collide:– Subtract and recurs – Small number of collisions converges in few iterations
• Every iteration needs new random hashing:– Permute time domain signal permute frequency domain
– is invertible mod : ’ = / ′ =
– Permutation : mod
![Page 16: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/16.jpg)
Algorithm• Iteration i :
– ∝ /2– Permutespectrum: ’ = /
– = ‐point DFT ′ = Subsample ′ ∗– Recover locations and values of large frequencies
• Each iteration takes O( ) time.• iterations • Total time :
![Page 17: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/17.jpg)
Experiments(exactly k‐sparse algorithm)
![Page 18: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/18.jpg)
Setup
• Similar to earlier work:– Random 0‐1 k‐sparse vectors – Fix n, vary k– Fix k, vary n
![Page 19: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/19.jpg)
Experiments
![Page 20: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/20.jpg)
Experiments, ctd
![Page 21: PiotrIndyk Dina Katabi Eric Price MIThaitham.ece.illinois.edu/Papers/sFFT_D.pdf · – Practice: faster runtime than FFT. Our results • Randomized algorithms, with constant probability](https://reader034.vdocument.in/reader034/viewer/2022051920/600daa6c51123d0f0d1f295a/html5/thumbnails/21.jpg)
Conclusions• Sparse FFT with running times :
– for exactly sparse case– for approximately sparse case– Improves over FFT for
• Significant improvement in practice
• time for approximately sparse signals?• Not clear: log / samples needed, extra log for FT