Week 11 Spectral TV and Convex analysis Guy Gilboa Course Topics: Some basic definitions in convex analysis. New research on Spectral TV. Guy Gilboa, Technion2 Classical Fourier filtering example Guy Gilboa, Technion3 Classical Signal Processing (Fourier) Positive features: Decomposition (transform) into a better representation, orthonormal basis. Filtering in the transform space simple amplification or attenuation of coefficients. Spectrum plot visualization of active frequencies, L2 energy is preserved Perseval identity. Linearity forward and inverse transforms are linear. A well established mathematical theory and fast computational methods. Known drawbacks : Does not handle well discontinuities and spatially local features, not an adequate basis for images.. Guy Gilboa, Technion4 Variational Spectral Processing Decomposition (transform) into a better representation, orthonormal basis. Filtering in the transform space simple amplification or attenuation of coefficients functions. Spectrum plot visualization of active generalized- frequencies, Perseval-type rule. Linearity forward and inverse transform are is linear. A well established mathematical theory and fast computational methods. Not yet.. Guy Gilboa, Technion5 TV spectral representation [G., SIAM-IS, 2014] Let u(t) be the TV-flow solution at time t with u(0)=f. TV- flow f t S(t)f... t H(t) 1 0 S(t) Guy Gilboa, Technion6 Nonlinear eigenvalue problem Linear problem (L linear operator) General operator T: A convex functional J(u) induces an operator p(u) by its subgradient: Guy Gilboa, Technion7 Understanding a regularizer is knowing its eigenfunctions [Alter-Caselles-Chambolle-2003]. My view : What are the TV eigenfunctions? Guy Gilboa, Technion8 Why do we get a delta in time for eigenfunctions ? Guy Gilboa, Technion9 Ideal low-pass-filter (LPF) eigenvalue Guy Gilboa, Technion10 Standard possible filters (borrowing the names from classical signal processing) Guy Gilboa, Technion11 1D Decomposition Example Guy Gilboa, Technion12 Application Guy Gilboa, Technion13 TV Band-Pass and Band-Stop filters fS(t) TV Band-stopTV Band-pass Old man Old man close up, original Old man 2 modes attenuated Wavelet vs. Spectral-TV decomposition f Haar WaveletsSpectral TV Guy Gilboa, Technion18 Haar Wavelets vs. Spectral TV WaveletSpectral TV Guy Gilboa, Technion19 Texture analysis and processing in the spectral TV domain with Dikla Horesh Guy Gilboa, Technion20 Spatially varying texture Perspective Lighting Combination Goal decompose textures which are gradually varying in scale, contrast or lighting. Scale change Perspective Lighting Spatially varying contrast and scale Guy Gilboa, Technion22 Input f(x) T(x) What happens for a natural image? Guy Gilboa, Technion23 Proposed result How can we use it to separate? Classical TV-G separation at some cutoff scale Guy Gilboa, Technion24 Time of maximal value of phi(t;x) for each pixel x Input Algorithm Input image Spectral decomposition Max response time Surface Fitting Take Max in each pixel Separation Bands Separated image layers Values in percentiles are taken for surface fitting Separation surface Band width taken for separationSeparation bands Input image Example 1 Decomposition example (perspective) Maximal phi response Proposed Input Rolling-Guidance-Filter [*] [*] Zhang et al, ECCV-2014 Guy Gilboa, Technion27 Desired textureStructure Input Texture enhancement Application: Texture enhancement enhanced Input Attenuated Application: Texture enhancement (2) Desired textureStructure Input Enhancement 2 enhanced Input Attenuated Guy Gilboa, Technion31 Michael Moellers texture transfer Guy Gilboa, Technion32