Predicting Wavelet Coefficients Over Edges Using Estimates

Based on Nonlinear Approximants

Onur G. Guleryuzoguleryuz@erd.epson.comEpson Palo Alto Laboratory

Palo Alto, CAgoogle: Onur Guleryuz

Overview

Topic: Wavelet compression of piecewise smooth signals with edges.(piecewise sparse)

Benchmark scenario:

Piecewise smooth signal Erase all high frequency wavelet coefficients

Predict erased datamse?

Outline:•Background and Problem Statement•Formulation•Algorithm•Results

More than what I am doing, it’s how I am doing it.

NotesQ: What are edges? (Vague and loose) A: Edges are localized singularities that separate statistically uniform regions of a nonstationary process.Caveats: This method is not:• edge/singularity detection, • convex (and therefore not POCS), • solving inverse problems under additive noise (wavelet-vaguelette), • an explicit edge/singularity model.

This method is: • a systematic way of constructing adaptive linear estimators, • an adaptive sparse reconstruction, • based on sparse nonlinear approximants (non-convex by design), • a model for non-edges (sparsity/predictable detection).

No amount of looking at one side helps predict the other side.

Wavelet Compression in 1-D and 2-D

Wavelets of compact support achieve sparse decompositions

A. Cohen, I. Daubechies, O. G. Guleryuz, and M. T. Orchard, ``On the importance of combining wavelet-based nonlinear approximation with coding strategies,'' IEEE Trans. Info. Theory}, vol. 48, no. 7, pp. 1895-1921, July 2002.

1-D

M. N. Do, P. L. Dragotti, R. Shukla, and M. Vetterli, ``On the compression of two-dimensional piecewise smooth functions,'‘ Proc. IEEE Int. Conf. on Image Proc. ICIP ’01, Thessaloniki, Greece, 2001.

2-DToo many wavelet coefficients over edges

(Need to reduce)

Current Approaches“1”: Modeling higher order dependencies over edges in wavelet domain.

•F. Arandiga, A. Cohen, M. Doblas, and B. Matei, ``Edge Adapted Nonlinear Multiscale Transforms for Compact Image Representation ,'‘ Proc. IEEE Int. Conf. Image Proc., Barcelona, Spain, 2003.

•H. F. Ates and M. T. Orchard, ``Nonlinear Modeling of Wavelet Coefficients around Edges,'‘ Proc. IEEE Int. Conf. Image Proc., Barcelona, Spain, 2003.

•J. Starck, E. J. Candes, and D. L. Donoho, ``The Curvelet Transform for Image Denoising,'‘ IEEE Trans. on Image Proc., vol. 11, pp. 670-684, 2002.

•P.L. Dragotti and M. Vetterli, ``Wavelet footprints: theory, algorithms, and applications,'‘ IEEE Trans. on Sig. Proc., vol. 51, pp. 1306-1323, 2003.

•M. Wakin, J. Romberg, C. Hyeokho, and R. Baraniuk, ``Rate-distortion optimized image compression using wedgelets,'‘ Proc. IEEE Int. Conf. Image Proc. June 2002.

“2”: New Representations.

……

Translation/rotation invariance is an issue.

Best linear representations are given by overcomplete transforms.

(Reduce by prediction)

(Don’t create too many)

Q: What are Overcomplete Transforms?

•Spatial DCT tilings of an Image

…

image-wide, orthonormal transform

G1 G2 … GM

Image arranged in a (Nx1) vector x, are (NxN)G i

Example: Translation invariant, overcomplete transforms

Sparse Decompositions and Overcomplete Transforms

G1 sparse portions nonsparse portions

No single orthonormal transform in the overcomplete set provides a very sparse decomposition.

G2

GM

…

1V

MV

…

1

11

S

I

GG

G

,...}2,4{1 V

image

Issues with Overcomplete Trfs

.

.

....1 MGGimageCompression angle:

Thresholding based Denoising:sparse portions nonsparse portions

G1

GM

…

)(1 TV

)(TV M…

image(x)

1IG

MIG

… xGGGG MI

MTII

TI )]...(1[ 11

remove the insignificant coefficients and the noise that they contain

)( xDT

DCC’02Onur G. Guleryuz, "Nonlinear Approximation Based Image Recovery Using Adaptive Sparse Reconstructions and Iterated Denoising: Part I - Theory“, “Part II – Adaptive Algorithms,” IEEE Transactions on Image Processing, in review.

http://eeweb.poly.edu/~onur

1

0

xx

x

1

0

x̂x

y

Fill missing information with initial values, T=T .

Denoise image with hard-threshold T.

Enforce available information.

T=T-dT

0

)( yDT

Nonlinear Approximation and Nonconvex Image Models

availablesample

missingsample

Sample coordinates for a two sample signal Recovery transform coordinates

Find the missing data to minimize

yGGGGy MI

MTII

TI

T )...( 11 ))()...(( 1 TVTV M

1

m̂inx

Assume single transform

Underlying Estimation Method

0)1(0

yyDy TT

1

m̂inx

))()...(( 001 dTTVdTTV M

1)1(1

yyDy TT

1

m̂inx

))()...(( 221 TVTV M

)( 01 dTTT

yGGGGy MI

MTII

TI

T )...( 11

There is method to the denoise, denoise, …, denoise madness.

•No explicit statistical modeling.•Systematic way of generating adaptive linear estimators.•It doesn’t care about the nonsparse portions of transforms (must identify sparse portions correctly)•Sparse predictable.•Relationships to harmonic analysis.

DCT2=shift(DCT1) DCTM=…

Modeling “Non-Edges” (Sparse Regions)

smooth

smooth

edgeDCT1

yGGGGy MI

MTII

TI

T )...( 11 1

m̂inx

I don’t care how badly the transform I am using does over the edges.I determine non-edges aggressively.

Algorithm

Onur G. Guleryuz, ``Weighted Overcomplete Denoising,‘’ Proc. Asilomar Conference on Signals and Systems, Pacific Grove, CA, Nov. 2003.

Fill missing information (high frequency wavelet coefficients)with initial values (0), T=T .

Denoise image with hard-threshold T.

Enforce available information (low frequency wavelet coefficients).

T=T-dT

0

)( yDT

I use DCTs and a simple but good denoising technique:http://eeweb.poly.edu/~onur

Test Images

Teapot (960x1280) Lena (512x512)

Graphics (512x512)

Bubbles (512x512)

Pattern (512x512)

Cross (512x512)

I admit, you can do edge detection on this one

Implementation

1: l-level wavelet transform (l=1, l=2)

2: All high frequency coefficients set to zero (l=1 half resolution, l=2 quarter resolution)

3: Predict missing information

4: Report PSNR=10log10(255*255/mse)

Results on Graphics

Graphics, l=1 Graphics, l=2

30.48dB to 51dB 27.15dB to 37.44dB

Results on Bubbles

Bubbles, l=1 Bubbles, l=2

33.10dB to 35.10dB 29.03dB to 30.14dB

Bubbles crop, l=1

Unproc.: 30.41dB Predicted: 33.00dB

magnitude info.location info

Bubbles crop, l=2

Unproc.: 26.92dB Predicted: 28.20dB

Pattern crop, l=1

Unproc.: 25.94dB Predicted: 26.63dB

still a jump

Holder exponentextrapolation, step edge assumption, edge detection, etc., aren’t going to work well here.

Cross crop, l=1

Unproc.: 18.52dB Predicted: 18.78dB

Holder exponentextrapolation, step edge assumption, edge detection, etc., aren’t going to work well here.

PSNR over 3 and 5 pixel neighborhood of edges (l=1)

3 pixel neigh. 5 pixel neigh. overall

Graphics Unprocessed 18.23 dB 20.22 dB 30.48 dB

Predicted 39.00 dB 41.00 dB 51.00 dB

Bubbles U 24.61 dB 26.52 dB 33.10 dB

P 28.56 dB 30.29 dB 35.10 dB

Pattern U 20.46 dB 22.02 dB 27.04 dB

P 22.39 dB 23.83 dB 27.48 dB

Cross U 16.88 dB 17.44 dB 18.72 dB

P 18.32 dB 18.52 dB 18.87 dB

+21 dB+21 dB

+2 dB+4 dB

+0 dB+1.5 dB

+0.5 dB+2 dB

Comments and Conclusion

• I will show a few more results.• Around edges, magnitude and location distortions.• Instead of trying to model many different types of edges, model non-edges as sparse (same algorithm handles all varieties).• Early work 1: Interpolation in pixel domain may give misleading PSNR numbers for two reasons. • Early work 2: Hemami’s group and Vetterli’s group have wavelet domain results (based on Holder exponents), but not on same scale.• You can implement this for your own transform/filter bank (denoise, available info, reduce threshold, …).

Results on Teapot

Teapot, l=1 Teapot, l=2

36.17dB to 41.81dB 32.54dB to 35.93dB

Teapot crop, l=1

Unproc.: 28.38dB Predicted: 34.78dB

Teapot crop, l=2

Unproc.: 25.10dB Predicted: ??.??dB

Results on Lena

Lena, l=1 Lena, l=2

35.26dB to 35.65dB 29.58dB to 30.04dB

Lena crop, l=1

Unproc.: 34.42dB Predicted: 35.03dB

Lena crop, l=2

Unproc.: 27.79dB Predicted: 29.83dB