ieee transactions on image processing, vol. …pvishalm/journal_pub/sheardec.pdf · ieee...

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 18, NO. 12, DECEMBER 2009 2673

Shearlet-Based DeconvolutionVishal M. Patel, Member, IEEE, Glenn R. Easley, Member, IEEE, and Dennis M. Healy, Jr.

Abstract—In this paper, a new type of deconvolution algorithm isproposed that is based on estimating the image from a shearlet de-composition. Shearlets provide a multidirectional and multiscaledecomposition that has been mathematically shown to representdistributed discontinuities such as edges better than traditionalwavelets. Constructions such as curvelets and contourlets sharesimilar properties, yet their implementations are significantly dif-ferent from that of shearlets. Taking advantage of unique proper-ties of a new M-channel implementation of the shearlet transform,we develop an algorithm that allows for the approximation inver-sion operator to be controlled on a multiscale and multidirectionalbasis. A key improvement over closely related approaches such asForWaRD is the automatic determination of the threshold valuesfor the noise shrinkage for each scale and direction without explicitknowledge of the noise variance using a generalized cross valida-tion (GCV). Various tests show that this method can perform sig-nificantly better than many competitive deconvolution algorithms.

Index Terms—Deconvolution, generalized cross validation,shearlets, wavelets.

I. INTRODUCTION

I N image restoration, the goal is to best estimate an imagethat has been degraded. Examples of image degradation in-

clude the blurring introduced by camera motion as well as thenoise introduced from the electronics of the system. In the casewhen the degradations can be modelled as a convolution oper-ation, the process of recovering the original image from the de-graded blurred image is commonly called deconvolution. Theprocess of deconvolution is known to be an ill-posed problem.Thus, to get a reasonable image estimate, a method of reducing/controlling noise needs to be utilized.

Wavelets are popular for image representation and are usedin a wide variety of image processing applications such as com-pression, and image restoration [1], [47]. The main reason forwavelets’ success can be explained by their ability to sparselyrepresent 1-D signals which are smooth away from point dis-continuities. By sparse representation, we mean that most of thesignal’s energy can be captured by a few of the transform co-efficients. This is quantified by the decay rate of the nonlinear

Manuscript received January 04, 2009; revised July 12, 2009. First publishedAugust 07, 2009; current version published November 13, 2009. The associateeditor coordinating the review of this manuscript and approving it for publica-tion was Dr. Luminita Vese.

V. M. Patel is with the Department of Electrical and Computer Engi-neering, University of Maryland, College Park, MD 20742 USA (e-mail:[email protected]).

G. R. Easley and D. M. Healy, Jr. are with the Department of Mathematics,University of Maryland, College Park, MD 20742 USA (e-mail: [email protected]; [email protected]).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TIP.2009.2029594

approximation error. Indeed, it can be shown that the best non-linear -term wavelet expansion (reconstruction from thelargest coefficients) for this type of signal has the rate of decaythat is the best achievable [3], [4]. It is understood that, the betterthe decay rate, the better the signal estimate from the noisy datawill be from that representation.

It is because of this optimality property of wavelet representa-tions that wavelet-based deconvolution routines have been pro-posed. However, wavelet representations are actually not op-timal for all types of images. Specifically, in dimension two, ifwe model images as piecewise smooth functions that are smoothaway from a edge,1 the standard 2-D wavelets do not reachthe best possible rate. In particular, the approximation error fora wavelet representation decays as as increases [4].As a result, denoising estimates based on 2-D wavelets tend tohave small unwanted artifacts and complex decision metrics orschemes need to be utilized to try to improve the quality of theestimate. Multidirectional representations such as shearlets [5],[6] provide nearly the optimal approximation rate for these typesof images (the optimal rate being as increases [7])and the corresponding denoising estimates do not suffer fromthe same types of artifacts [8]. Although related transforms suchas contourlets [9], [10], and curvelets [11]–[13] share similarproperties, in this work we utilize properties unique to an im-plementation of the shearlet transform that offer advantages forthe purpose of deconvolution.

The concept of using a sparse representation to achieve goodestimates for deconvolution has been suggested before (see forexample [14] and [15]). However, particular features concerningimplementations of such representations that contribute to per-formance presented here have not been previously considered.Our shearlet-based deconvolution has the unique ability for amultiscale and anisotropic regularization inversion to be donebefore noise suppression. Furthermore, for a given regulariza-tion parameter its adaptive noise suppression surpasses similarschemes. This is an important consideration since in some case itmay not be possible to find the optimal regularization parameter.

In the implementation stage, to deal with boundary effects,some concepts in the literature have centered around the idea ofnoise shrinkage either before or after the application of the de-convolution procedure (see [14] and [16]). However, to carry outsuch schemes effectively, one needs a transform that can be im-plemented in a nonrecursive formulation as is done in this workwith the shearlet transform. Otherwise, error estimates made byone set of coefficients will highly influence estimates made on adifferent but dependent set of coefficients. In addition, to be ef-fective in regularizing the approximate deconvolution process, anonsubsampled (redundant) transform should be utilized. Thisredundancy not only provides for more effective measurements

1� is the space of functions that are bounded and 2-times continuouslydifferentiable.

1057-7149/$26.00 © 2009 IEEE

2674 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 18, NO. 12, DECEMBER 2009

based on the use of auxiliary functions such the GCV function,but it also greatly aids in estimation [10], [17]–[19]. As will be-come apparent, these desired features are obtained by using anM-channel shearlet transform implementation.

A. Image Deconvolution Problem

Since a digitally recorded image is a finite discrete data set,an image deconvolution problem is formulated as a matrix inver-sion problem. Without loss of generality, assume the recordedarrays are of size . Let denote an array of sam-ples from a zero mean additive white Gaussian noise (AWGN)with variance . Given the arrays and , representingthe observed image and the image to be estimated, respectively,the matrix deconvolution problem can be described as

(1)

where , , and are column vectors representing thearrays , , and lexicographically ordered, and is the

matrix that models the blur operator. In the case when isa block-circulant-circulant-block matrix [20], the problem canbe described as

(2)

where , denotes circular convolution,and denotes the point spread function (PSF) of a linear space-invariant system. Equation (2) in the discrete Fourier transform(DFT) domain can be written as

(3)

where , , and are the2-D DFTs of , , , and , respectively, for

. The conditioning of this system is determined by theratio of the largest to smallest magnitudes of the values. Typ-ically, contains values at or near zero which makesthe system ill-conditioned.

In general, to regularize the inversion of the convolution op-erator, a representation that diagonalizes the convolution op-erator (matrix) is needed in order to appropriately control theapproximation. In particular, if is a block-circulant-circu-lant block matrix, is diagonalizable by a Fourier basis. Thismeans that an estimate of the image can be found by filteringthe diagonal components of the Fourier diagonalization of inorder to approximately invert . For instance, let bea filter that is nearly one when is large, is small when

is nearly zero, and such thatis defined everywhere. Then, a Fourier-based estimate can begiven as . This in turn meansthat the image is estimated from a Fourier representation.

However, if our image is considered as a piecewise smoothfunction that is smooth away from a edge, then the decay rateof the nonlinear approximation from a Fourier representation is

as increases. Yet, for this type of image the decayrate of the nonlinear approximation from a wavelet representa-tion is as increases. This means that estimating theimage from the perspective of removing noise, a wavelet basedestimate would perform better than the one from a Fourier basis.

In short, the ability to get a good estimate depends on balancinga representation that is effective for regularizing the inversionof the convolution operator and a representation that is effec-tive for estimating the image from colored noise by means ofthe approximate inversion of the operator. (See [15] for furtherdiscussion).

B. Historical Perspective

Deconvolution methods can be separated into two major cat-egories: direct and iterative.

Direct Methods: Some of these methods are based onfiltering the singular value decomposition (SVD) such asTikhonov, truncated SVD (TSVD), and Wiener filtering [21].Increased performance of such direct methods can be attributedto the inclusion of the wavelet-based estimators. One such tech-nique called the Wavelet-Vaguelette deconvolution (WVD) wasproposed in [14]. In this work, functionals called vaguelettesare used to simultineously deconvolve and compute the waveletcoefficients. However, the scheme does not provide goodestimates for all convolution operators. To overcome this lim-itation, Kalifa et al. proposed a wavelet packet based methodthat matches the frequency behavior of certain convolutionoperators [22]. Additional wavelet-based techniques have beenproposed in [16] and [23]–[25].

An improved hybrid wavelet based regularized deconvolu-tion algorithm that works with any ill-conditioned convolutionsystem was developed in [17]. This Fourier-Wavelet Regular-ized Deconvolution (ForWaRD) method employs Fourier-do-main regularized inversion followed by wavelet-domain noiseshrinkage to minimize the distortion of spatially localized fea-tures in the image. An extension in terms of curvelets, known asForCuRD, was proposed in [26].

Iterative Methods: Some of the better-known basic iter-ative methods are the Conjugate Gradient algorithm [21],Richardson-Lucy [27], [28], and Landweber [29]. Many exten-sions and improvements over these methods have been madethat include the use of wavelets, or other sparse representationssuch as curvelets. Some of these are [18], [30]–[40]. Additionaltechniques may be found within these references. Note someof the these promising techniques make use of one-norm trans-form-domain sparsity promotion. Such methods seem to retainedge information well.

Among the direct methods, the local polynomial approxima-tion (LPA) algorithm [41] outperformed some of the best ex-isting deconvolution methods such as [17] and [42] in termsof improvement in signal-to-noise-ratio and was established tobe state-of-the-art. In this work, we will only focus on directmethods for comparison because some applications may desiredirect methods, and also since some of these iterative methodscan use the estimates provided from such techniques as the ini-tial starting point.

C. Shearlet-Based Deconvolution

In this paper, we propose a new approach to image decon-volution that balances a Fourier domain regularized inversionand a shearlet-domain-based noise reduction. An undecimatedshearlet transform decomposes an image into various frequency

PATEL et al.: SHEARLET-BASED DECONVOLUTION 2675

regions whose supports are contained in a pair of trapezoidal re-gions symmetric with respect to the origin and directionally ori-ented. This aspect provides the unique ability to adaptively reg-ularize the Fourier inversion in a directionally oriented fashionwhen the blurred image is first projected onto a shearlet do-main. Following the regularized inversion, the colored noise isthen suppressed using a shearlet domain based thresholding.To improve the estimating capability, we derive a generalizedcross validation function to find the optimal shearlet domainshrinkage without explicitly calculating the noise variance.

D. Paper Organization

In Section II, we give a brief introduction to the shearlet trans-form. In Section III, we describe the use of generalized crossvalidation for shearlet thresholding for colored noise. In Sec-tion IV, we discuss details about the proposed deconvolutionalgorithm. In Section V, we show some of the simulation re-sults, and present the concluding remarks in Section VI.

II. SHEARLET TRANSFORM

In this section, we briefly describe a recently developed mul-tiscale and multidirectional representation called the shearlettransform [6].

Consider the 2-D affine system

where

is a product of a shearing and an anisotropic dilation matrix for. The generating functions are such that, for

(4)

where is a continuous wavelet for which with, and is chosen so that

, , , withon ( 1, 1). Then any admits the representation

for , , and . The operator defined by

is known as the continuous shearlet transform of .The shearlet transform is a function of three variables: the scale

, the shear and the translation . In the frequency domain,has support in the set

Hence, each element has support on a pair of trapezoids,at various scales, symmetric with respect to the origin and ori-ented along a line of slope .

The collection of discrete shearlets is described by

where

For the appropriate choices of , the discrete shearlets form aParseval frame (tight frame with bounds equal to 1) for[43], i.e., they satisfy the property

The discrete shearlets described above provide a nonuniformangular covering of the frequency plane when restricted to thefinite discrete setting for implementation. Thus, it is preferred toreformulate the shearlet transform with restrictions supported inthe regions given by

and . Specifi-cally, define

where ,and . In addition, we assume that

for

and, for each

for

Let

and choose to satisfy

for , where denotes the indicator function of theset . With the functions and as above, we deduce the fol-lowing result.

Theorem 1 ([6]): Let and. Then the collection of shearlets

together with

is a Parseval frame for , where .


Based on and , filters and can be found so that

and can be computed as

where are the directionally-oriented filters.To simplify the notation, we suppress the superscript andabsorb the distinction between and 1 by re-indexing theparameter so that it has double the cardinality. An M-channelfilterbank implementation whose filters correspond to canbe done by using the techniques given in [44]. As a consequence,its implementation has a complexity of for an

image.Notice that, just as in the continuous version, each element

is supported on a pair of trapezoids, and each trapezoid iscontained in a box of size approximately satisfying aparabolic scaling property. Their supports become increasinglythin as and the elements exhibit highly directionalsensitivity since they are oriented along lines with slope givenby . These properties contribute to being able to establishthe following theorem.

Theorem 2 ([45]): Let be away from piecewisecurves, and let be the approximate reconstruction of usingthe largest coefficients in the shearlet expansion. Then

The significance of this result is that a shearlet-based estimateyields a MSE approximation rate of as , whereis the noise level of the noisy image [12]. (This is achieved bychoosing a threshold so that one reconstructs from the largest

noisy shearlet coefficients.) Similarly, one obtainsthe MSE approximation rate of wavelet thresholding as for

.The shearlet transform has similarities to the curvelet trans-

form and the contourlet transform. Shearlets and curvelets, infact, are the only two systems which are known mathematicallyto provide the rate of using the largestcoefficients for images described as away from piecewise

curves. The spatial-frequency tilings of curvelet and shearletrepresentations are completely different theoretically, yet theimplementations of the curvelet transform corresponds to es-sentially the same tiling as that of the shearlet or contourlettransform.

Alternative discrete shearlet decompositions can be createdby varying the support of the mother wavelet (which amountsto changes in ) and changing the dilation matrices and .Such changes produce different spatial-frequency tilings com-posed of regions of support that are restricted to pairs of var-ious trapezoidal regions. When implemented in an undecimatedform, the shearlet transform will produce a highly redundant de-composition consisting of the total number of paired trapezoidalregions considered.

An important advantage in the use of this redundant shearlettransform implementation for deconvolution is that it allows oneto independently estimate each directionally-oriented frequencyband with different amounts of regularization. This has not been

done before and cannot be done by the current curvelet or con-tourlet transform implementations.

III. GENERALIZED CROSS VALIDATION (GCV) FOR

SHEARLET THRESHOLDING

In this section, we describe a shearlet-thresholding schemebased on a GCV function for the purpose of noise reduction[46]. One of the major advantages of this GCV method is thatit obtains nearly the optimal thresholding without knowing thenoise variance. It depends only on the data and automaticallyadjusts the shrinkage parameter according to the data. A sim-ilar GCV method for wavelet thresholding has been proposedin [48]–[50]. Note that, although we are suggesting the use ofa GCV function, is also it possible to adapt the new SURE ap-proach [51] for this task.

Suppose

(5)

where the vectors , and represent respectively the ob-servation, the original image and the colored noise that is as-sumed to be second order stationary (i.e., the mean is constantand the correlation between two points depends only on the dis-tance between them). Corresponding to a threshold , define thesoft-threshold function to be equal to if

and zero otherwise. We will show that nearly optimalthreshold values can be obtained by finding the values min-imizing a GCV function which is dependent on each scale anddirection .

Just as in the case of wavelets, to obtain results similar tothose in [50], it is not necessary for the shearlet coefficients tobe uncorrelated at any moment; however, it is necessary thatthe noise be second order stationary [49]. If the noise process

is stationary, then using the multiscale and multidirectionalstructure of shearlets, we obtain the following lemma.

Lemma 1: If represents a shearlet coefficient of arandom vector at scale , direction , and location , then thevariance of this coefficient, , depends only onthe scale and direction .

Proof: It follows from the fact that we are using a filterbank with appropriate directional filters and if is a discretestationary random process which is an input to a shift-invariantfilter, , corresponding to scale and direction then theoutput is a convolution of with which is also stationary[52].

By Lemma 1, the shearlet transform of stationary correlatednoise is stationary within each scale and directional component.We can use this property to choose a different threshold foreach resolution and directional component. If represents thevector of shearlet coefficients of at scale and direction , thenwe can write

(6)

where is the number of shearlet coefficients on scale anddirection , is the total number of shearlet coefficients, and

(7)


Since all the components in (6) are positive, minimizing themean squared error or risk function is equivalent to min-imizing for all and . An argument similar to thatused in [50] leads to the following GCV functions:

(8)

where is the total number of shearlet coefficients that werereplaced by zero. We now have the following result.

Theorem 3: The minimizer of is asymptoticallyoptimal for the minimum risk threshold for scale anddirectional component .

Thus, by using the values that minimize for eachand , a shearlet-based denoised estimate will likely be close

to the ideal non-noise corrupted image.An important feature about the use of our nonsubsampled

shearlet transform implementation is that it facilitates the use ofasymptotic methods such as those based on GCV functions. Asubsampled transform implementation would cause the numberof coefficients to decrease as the levels progress, so that thresh-olds found by minimizing the individual GCV functions will beless likely to correspond to the actual threshold values that min-imize the risk functions for each frequency band.

IV. SHEARLET-BASED DECONVOLUTION

Having established a method for obtaining a good image es-timate when the image is corrupted by colored noise, let us nowfocus on how we are to use this method as part of a deconvo-lution routine. Since our blurring model is described by (2), asuitable pseudo-inverse estimate can be found by regularizingthe convolution operator from a discrete Fourier basis. Usingthe regularized inverse operator

(9)

for some regularizing parameter , an image estimate inthe Fourier domain is given by

(10)

for . This type of regularization ap-plied is often referred to as Tikhonov-regularization [53]. Whenan estimate of the power spectral density (PSD) can be accu-rately determined from a method such as that proposed in [54],a Wiener-based solution can be found by using

(11)

where and is the estimated PSD of the imagefor .

Taking advantage of the shearlet decomposition, we can adap-tively control the regularization parameter to be the best suitedof each frequency supported trapezoidal region. Let denote

Fig. 1. Frequency support of the shearlets for different values of � and �.

the DFT of the shearlet filter for a given scale and direc-tion . The shearlet coefficients of an estimate of the image fora given regularization parameter can be computed in theFourier domain as

for .The remaining aspect of the deconvolution problem is trans-

formed into a denoising problem in the presence of colorednoise. This can be dealt with by thresholding the estimatedshearlet coefficients using the GCV determined previouslywithout having to know the noise variance explicitly. An im-portant advantage in using the GCV is that after the Fourierregularized inversion (FRI), the method automatically adjuststhe shrinkage parameter according to the data. We summarizethe shearlet based deconvolution method in Fig. 2.

Let denote the result of the inversion of the shearlet trans-form after the shearlet coefficients of have been thresh-olded. We want to choose an that minimizes the shearlet-basedmean-squared-error (MSE) . However, since is un-known, is chosen to be a minimizer of the cost function

where , is the mean of , andthe sum is taken over all values of and from to

inclusive. In other words, we choose such thatthe estimate agrees with the observation based on weighing

counter-balanced byan approximation to for each frequency index

. This is just an extension (shearlet-based estimateversus wavelet-based estimate) of the cost function originallysuggested in [17]. This optimization function assumes the noisevariance to be known. This is not a problem since the noisevariance can easily be estimated by using a median estimatoron the finest scale of the wavelet coefficients of [4], [47].


Fig. 2. Fourier-shearlet regularized deconvolution.

Fig. 3. Images used in this paper for different experiments. (a) Cameramanimage, (b) Barbara image, and (c) Lena image, (d) Peppers image.

In this case, there is a great advantage in using the GCV forthe shearlet thresholding as the variance of the colored noise ateach location and scale dependent on does not have to be esti-mated. In addition, a GCV-based thresholding routine producesbetter results over schemes based on estimating the standard de-viation of the noise throughout the decomposition.

If we define for, then the optimal for each

TABLE IISNR FOR DIFFERENT EXPERIMENTS

thresholded shearlet coefficient can be found by mini-mizing the cost function equal to

where and is the inverse DFTof .

The use of this optimization function to find has been shownto be satisfactory with many of the examples tested. The L-curvemethod could be adopted to estimate (see [21] for details onthe method). This could prove to be more reliable and does notrequire any estimate of the noise variance. It is also possible touse the optimization functionwhere denotes the regularized Tikhonov inverse of and

denotes the identity matrix. Such an optimization functionweighs the fidelity of the estimate against the given dataand inversely weighs it against a measure of how far away theregularized inverse operator is from an idealized inversion oper-ator. This generalized cross validation function can be derivedby using similar arguments to that given in [46].


Fig. 4. � and �� as functions of the threshold � for different scales and directions from Experiment 1. Solid (blue) line corresponds to the �� anddotted (red) line corresponds to � . (a) � � �� , (b) � � �� , (c) � � �� .

We summarize the main steps of the shearlet-based deconvo-lution algorithm as follows.

Shearlet-based Deconvolution Algorithm

Given , for some and .• Use the shearlet filter and apply the regularized filter

(10) or (11) to to obtain .• Apply the GCV based shearlet shrinkage to to

obtain .

Repeat process for a different value of until isminimized for each and .

After each shearlet coefficient that minimizes isfound, form the final estimate by applying the inverse shearlettransform.

The values that minimize can be found by using ei-ther a sequence of possible values or by using a minimizationroutine.

Although we have described the most general case of regu-larizing each shearlet coefficient separately, in some cases forefficiency it may be preferred to use a common regularizationparameter . In such a case, the algorithm is implemented usingthe coast function instead of .

V. EXPERIMENTAL RESULTS

In this section, we present results of our proposed algo-rithm and compare them with some of the recent multiscalewavelet and wavelet-like deconvolution methods describedin [17], [41] and [26]. The implementation of the ForWaRDalgorithm is available at www.dsp.rice.edu/software and theimplementation of the LPA-ICI algorithm is available atwww.cs.tut.fi/~lasip/2D. In these experiments we will use theimprovement in signal-to-noise-ratio (ISNR) to measure theperformance of the routines tested using the images shown inFig. 3. The ISNR is defined as

For an image of size , the BSNR is defined in decibels as

where denotes the mean of .For all shearlet transform implementations, we used 1, 8, 8,

16, and 16 directions in the scales from coarse to fine whichcorresponds to the same decomposition tested in [6]. We applythe GCV based shrinkage to the outputs from each of the 48filters except the output corresponding to the coarsest scale. Ex-periments have shown that increasing the number of directionsevery scale usually results in better estimates.

In the case when wavelets are used for image denoising,it was shown in [55] that a Wiener-based wavelet shrinkagefilter typically improves upon the mean square error perfor-mance over that of hard/soft thresholding. By Wiener-basedshrinkage, we mean to weigh the wavelet coefficientsas , where are the waveletcoefficients from another denoised estimate, and arescale-dependent regularization parameters. The performanceof the proposed method is improved by using a similar Wienershrinkage filter. In this case, the shearlet coefficients with aslightly different decomposition (three decomposition levels)are filtered using the initial shearlet-based estimate. Severalexperiments have shown that the final estimate is mostly drivenby how successful the initial estimate is, so that even the useof a wavelet decomposition instead of an alternate shearletdecomposition can provide just as an effective estimate.

In the first set of tests, we consider the setup of [17], where aCameraman image is blurred by a 9 9 uniform box-car blur.The AWGN variance, , is chosen with a BSNR of 40 dB.A comparison of different methods in terms of ISNR is shownin Table I under the Experiment 1 column. The shearlet-basedmethod yields a value 7.89 dB which is better than the valuesobtained by any of the other methods.

In Fig. 4, we display a few of the and curvesfor different scales and directions obtained from Experiment 1.In [17], after the Fourier shrinkage, the leaked colored noisevariance was estimated at each scale and was used to shrink


Fig. 5. Minimum values obtained by minimizing � (black), and �� (dot-dash) values along with � (gray) values obtained by the method described in[17], for different scales and directions from Experiment 1. As can be seen from the figure, minimum values of �� are approximately equal to the minimumvalues of � . (a) At scale � � �, there are 16 directions. (b) At scale � � �, there are 16 directions. (c) At scale � � �, there are eight directions. (d) At scale� � �, there are eight directions.

the wavelet coefficients. Similarly, we can estimate the colorednoise variance, , at different scales and directions at theoutput of the shearlet filter bank as follows:

The threshold values are determined by ,where is a scale and direction dependent threshold [4], [17].For comparison, the estimated are plotted in Fig. 5 alongwith the actual minimum values obtained by minimizing the

and functions for experiment 1. Figs. 4 and 5 in-dicate that both and have approximately the sameminimum values. However, in some cases, the estimatedvalues are very different than the values obtained by minimizingthe and curves.

In Fig. 6, we show the signal-to-noise-ratio (SNR) perfor-mance of the shearlet-based deconvolution (black-line) com-pared to ForWaRD (dotted-line) and ForCuRD (gray-line) asa function of blur SNR (BSNR). In this illustration, we usedthe 9 9 box-car blur on the Lena image shown in Fig. 3(c).As explained previously, an estimate based on a shearlet orcurvelet decomposition decays faster than that of an estimatebased on a wavelet decomposition as a function of noise level

Fig. 6. SNR performance of ForSURE (black-line) compared to ForCuRD(gray-line) and ForWaRD (dotted-line) as a function of BSNR.

for images that are smooth away from edges. Fig. 6 dis-plays a similar correspondence in decay rates for the proposedshearlet-based deconvolution and the ForCuRD scheme overthe wavelet-based deconvolution scheme (ForWaRD). Sincethe performance is measured in terms of SNR instead of MSE,it is expected that shearlet and curvelet-based estimates willdecay slower as a function of noise level given in terms ofBSNR.

In the second set of experiments performed over the Cam-eraman image, we replicate the experimental setup of [42].


Fig. 7. Details of the image deconvolution experiment with a Barbara image. (a) Original image. (b) Noisy blurred image, � � � dB. (c) ForWaRDestimate, �� dB. (d) ForCuRD estimate, �� dB. (e) LPA-ICI algorithm, �� dB. (f) Shearlet-based estimate, ��

�� dB.

The point spread function of the blur operator is given by:, for , and

the noise variances are and . The SNR im-provements are summarized in Table I under the Experiment


Fig. 8. Details of the image deconvolution experiment with a Peppers image. (a) Original image. (b) Noisy blurred image, �� dB. (c) Regularizedinversion estimate, �� dB. (d) ForWaRD estimate based on result shown in (c), �� dB. (e) LPA-ICI estimate based on result shown in(c), �� dB. (f) Shearlet-based estimate based on result shown in (c), �� dB.

2 and Experiment 3 columns, for and , re-spectively. Again, the shearlet-based deconvolution algorithmoutperforms the other methods in terms of ISNR.

In the third set of tests, the original image of Lena is blurredby a Gaussian PSF defined as

for , where is a normalizing constant en-suring that the blur is of unit mass, and is the variance that

determines the severity of the blur. In this experiment we choseand the noise variance, , with a BSNR of 40 dB. We

report the simulation results under the Experiment 4 column ofTable I. Again, our proposed method outperforms the best per-forming methods known for this problem setup.

In the fifth experiment, we use the blur filter considered in[41]. The original image of Barbara is blurred by a 5 5 sepa-rable filter with weights [1, 4, 6, 4, 1]/16 in both the horizontaland vertical directions and then contaminated with AWGN with

. The details of the images obtained by the differentmethods are shown in Fig. 7. Again, the shearlet-based algo-


Fig. 9. Plots of �� ,��, and �� forvarious values of are displayed as a black line, a gray line, and a dash-dottedline, respectively. The output values have been rescaled for illustration purposes.The locations of their minimal value are marked with a circled “X”.

rithm performs the best in terms of ISNR and captures the de-tails better than any of the other methods.

In light of the robustness to noise of the shearlet-basedmethod showed in Fig. 6, we replicated the set-up similar tothat given in [17]. In this case, the Peppers image is blurred bya 9 9 uniform box-car blur and the AWGN added is suchthat the dB. We tested the ForWaRD method

dB , the ForCuRD method dB ,the LPA-ICI method dB , and the shearlet-basedmethod dB using the same regularizationparameter (not necessarily optimal) for each routine (experi-ment 6). Close-ups of some of the results are shown in Fig. 8.The Fourier regularized inversion estimate used in all threealgorithms is shown in Fig. 8(c). This experiment presents animportant comparison in robustness to noise suppression andan indication of the shearlet-based algorithm’s high defaulttolerance level when the regularization parameter is not chosenoptimally.

Plots of the validation functions andare shown for various values of in Fig. 9.

For comparison, a plot of is also given. In the exper-iment set up for this comparision, we used the Peppers imageblurred by a 9 9 uniform box-car blur and the AWGN addedwith corresponding dB. The results give an in-dication that the validation function

can be just as effective as for finding estimatesof the optimal when the standard deviation of the noise is notestimated.

VI. CONCLUSION

In this work, we have proposed an effective shearlet-baseddeconvolution algorithm which utilizes the power of a Fourierrepresentation to approximately invert the convolution operatorand a redundant shearlet representation to provide an imageestimate. The multiscale and multidirectional aspects of theshearlet transform provide a better estimation capability overthat of the wavelet transform or wavelet-like transforms forimages exhibiting piecewise smooth edges. In addition, we have

adapted a method of automatically determining the thresholdvalues for the shearlet noise shrinkage without knowing thenoise variance by using a generalized cross validation function.Demonstrations show this method to outperform many of thestate-of-the-art methods that have been previously comparedto the ForWaRD algorithm in the literature and points thedirection of future research in this area to the utilization of suchmultiscale and multidirectional representations as shearlets.

A new multichannel implementation of the shearlet trans-form can be utilized that commutes with periodic convolution sothat may be projected onto a shearlet domain before es-timating the inversion of the convolution operator. This allowsfor the regularization of the approximate inversion operator tobe controlled on a multiscale and multidirectional basis. In thecases where the convolution operator is well behaved (see [14]for details) so that inversion of the convolution operator doesnot need approximating, this technique can be adapted so as toprovide a Vaguelette-Shearlet Decomposition (VSD) estimatesimilar to the Vaguelette-Wavelet Decomposition estimate pro-posed in [16]. Likewise, in the case of appropriately behavedconvolution operators, this technique could be easily adapted toprovide a Shearlet-Vaguelette Decomposition based estimate.

Several future directions of inquiry are possible consideringour new approach. Many complicated techniques that go beyondtrimming wavelet coefficients by a threshold parameter have beproposed, such as methods based on the hidden Markov models[56]. It is very likely that utilization of such techniques adaptedto the shearlet domain will provide an even greater performancewhen combined to a deconvolution routine. It would also be ofinterest to apply the methods developed here to blind deconvo-lution when the knowledge of the point spread function is notassumed.

ACKNOWLEDGMENT

The authors would like to thank R. Baraniuk, R. Neelamani,and the anonymous reviewers for their many valuable commentsand suggestions which significantly improved this paper.

REFERENCES

[1] D. L. Donoho and I. M. Johnstone, “Adapting to unknown smoothnessvia wavelet shrinkage,” J. Amer. Statist. Assoc., vol. 90, no. 432, pp.1200–1224, 1995.

[2] D. L. Donoho and I. Johnstone, “Ideal spatial adaptation via waveletshrinkage,” Biometrika, vol. 81, pp. 425–455, Dec. 1994.

[3] D. L. Donoho, M. Vetterli, R. A. DeVore, and I. Daubechies, “Datacompression and harmonic analysis,” IEEE Trans. Inf. Theory, vol. 44,pp. 2435–2476, 1998.

[4] S. Mallat, A Wavelet Tour of Signal Processing. New York: Aca-demic, 1998.

[5] D. Labate, W.-Q. Lim, G. Kutyniok, and G. Weiss, “Sparse multidi-mensional representation using shearlets,” in Proc. SPIE, Wavelet Ap-plications in Signal and Image Processing XI, Sep. 2005, vol. 5914, pp.254–262.

[6] G. R. Easley, D. Labate, and W.-Q. Lim, “Sparse directional imagerepresentations using the discrete shearlet transform,” Appl. Comput.Harmon. Anal., vol. 25, no. 1, pp. 25–46, Jul. 2008.

[7] D. L. Donoho, “Sparse components of images and optimal atomic de-composition,” Constr. Approx., vol. 17, pp. 353–382, 2001.

[8] G. R. Easley, D. Labate, and F. Colonna, “Shearlet-based total variationdiffusion for denoising,” IEEE Trans. Image Process., vol. 18, no. 2, pp.260–268, Feb. 2009.

[9] M. N. Do and M. Vetterli, “The contourlet transform: An efficient di-rectional multiresolution image representation,” IEEE Trans. ImageProcess., vol. 14, no. 12, pp. 2091–2106, Dec. 2005.


[10] A. L. Cunha, J. Zhou, and M. N. Do, “The nonsubsampled contourlettransform: Theory, design, and applications,” IEEE Trans. ImageProcess., vol. 15, no. 10, pp. 3089–3101, Oct. 2006.

[11] E. J. Candès and D. L. Donoho, “Curvelets—A surprisingly effec-tive nonadaptive representation for objects with edges,” in Curves andSurface Fitting, A. Cohen, C. Rabut, and L. L. Schumaker, Eds.Nashville, TN: Vanderbilt Univ. Press, 1999.

[12] J. L. Starck, E. J. Candès, and D. L. Donoho, “The curvelet transformfor image denoising,” IEEE Trans. Image Process., vol. 11, no. 6, pp.670–684, Jun. 2002.

[13] E. J. Candès, L. Demanet, D. L. Donoho, and L. Ying, “Fast discretecurvelet transforms,” Multiscale Model. Simul., vol. 5, no. 3, pp.861–899, 2006.

[14] D. L. Donoho, “Nonlinear solution of linear inverse problems bywavelet-vaguelette decomposition,” Appl. Comput. Harmon. Anal.,vol. 2, pp. 101–126, 1995.

[15] E. J. Candés and D. L. Donoho, “Recovering edges in ill-posed inverseproblems: Optimality of curvelet frames,” Ann. Statist., vol. 30, no. 3,pp. 784–842, 2002.

[16] F. Abramovich and B. W. Silverman, “Wavelet decomposition ap-proaches to statistical inverse problems,” Biometrika, vol. 85, no. 1,pp. 115–129, Oct. 1998.

[17] R. Neelamani, H. Choi, and R. G. Baraniuk, “ForWaRD: Fourier-wavelet regularized deconvolution for ill-conditioned systems,” IEEETrans. Image Process., vol. 52, no. 2, pp. 418–433, Feb. 2004.

[18] J. Starck, M. K. Nguyen, and F. Murtagh, “Wavelets and curvelets forimage deconvolution: A combined approach,” Signal Process., vol. 83,no. 10, pp. 2279–2283, 2003.

[19] R. R. Coifman and D. L. Donoho, “Translation invariant de-noising,” inWavelets and Statistics, A. Antoniadis and G. Oppenheim, Eds. NewYork: Springer-Verlag, 1995, pp. 125–150.

[20] M. K. Ng, R. H. Chan, and W. Tang, “A fast algorithm for deblurringmodels with Neumann boundary conditions,” SIAM J. Sci. Comput.,vol. 21, no. 3, pp. 851–866, 1999, (electronic).

[21] P. C. Hansen, Rank-Deficient and Discrete Ill-Posed Problems: Numer-ical Aspects of Linear Inversion. Philadelphia, PA: SIAM, 1998.

[22] J. Kalifa, S. Mallat, and B. Rouge, “Deconvolution by thresholding inmirror wavelet bases,” IEEE Trans. Image Process., vol. 12, no. 4, pp.446–457, Apr. 2003.

[23] I. M. Johnstone, G. Kerkyacharian, D. Picard, and M. Raimondo,“Wavelet deconvolution in a periodic setting,” J. Roy. Statist. Soc. BStat. Methodol., vol. 66, no. 3, pp. 547–573, 2004.

[24] M. Pensky and B. Vidakovic, “Adaptive wavelet estimator for non-parametric density deconvolution,” Ann. Statist., vol. 27, no. 6, pp.2033–2053, 1999.

[25] J. Fan and J.-Y. Koo, “Wavelet deconvolution,” IEEE Trans. Inf.Theory, vol. 48, no. 3, pp. 734–747, Mar. 2002.

[26] R. N. Neelamani, M. Deffenbaugh, and R. G. Baraniuk, “Texastwo-step: A framework for optimal multiinput single-output deconvo-lution,” IEEE Trans. Image Process., vol. 16, no. 11, pp. 2752–2765,Nov. 2007.

[27] W. H. Richardson, “Bayesian-based iterative method of image restora-tion,” J. Opt. Soc. Amer., vol. 62, pp. 55–59, 1972.

[28] L. B. Lucy, “An iterative technique for the rectification of observeddistributions,” Astron. J., vol. 79, pp. 745–754, 1974.

[29] L. Landweber, “An iterative formula for Fredholm integral equationsof the first kind,” Amer. J. Math., vol. 73, no. 3, pp. 615–624, Jul. 1951.

[30] J.-L. Starck, A. Bijaoui, and F. Murtagh, “Multiresolution supportapplied to image filtering and deconvolution,” CVIP: Graph. ModelsImage Process., vol. 57, pp. 420–431, 1995.

[31] J. M. Bioucas-Dias, “Bayesian wavelet-based image deconvolution: AGEM algorithm exploiting a class of heavy-tailed priors,” IEEE Trans.Image Process., vol. 15, no. 4, pp. 937–951, Apr. 2006.

[32] M. Elad, B. Matalon, and M. Zibulevsky, “Coordinate and subspaceoptimization methods for linear least squares with non-quadratic regu-larization,” Appl. Comput. Harmon. Anal., vol. 23, pp. 346–367, Nov.2007.

[33] G. Teschke, “Multi-frame representations in linear inverse problemswith mixed multi-contraints,” Appl. Comput. Harmon. Anal., vol. 22,pp. 43–60, 2007.

[34] I. Daubechies, G. Teschke, and L. Vese, “Iteratively solving linear in-verse problems,” Inv. Probl. Imag., vol. 1, no. 1, pp. 29–46, 2007.

[35] M. J. Fadili and J.-L. Starck, Sparse Representations-Based Image De-convolution by Iterative Thresholding, preprint, 2007.

[36] H. Takeda, S. Farsiu, and P. Milanfar, “Deblurring using regularizedlocally adaptive kernel regression,” IEEE Trans. Image Process., vol.17, no. 4, pp. 550–563, Apr. 2008.

[37] I. Daubechies, M. Fornasier, and I. Loris, Accelerated Projected Gra-dient Method for Linear Inverse Problems With Sparsity Constraints,preprint, 2008.

[38] C. Vonesch and M. Unser, “A fast thresholded Landweber algorithmfor wavelet-regularized multidimensional deconvolution,” IEEE Trans.Image Process., vol. 17, no. 4, pp. 539–549, Apr. 2008.

[39] I. Daubechies, M. Defrise, and C. De Mol, “An iterative thresholdingalgorithm for linear inverse problems with a sparsity constraint,”Comm. Pure Appl. Math., vol. 57, no. 11, pp. 1413–1457, 2004.

[40] P. de Rivaz and N. Kingsbury, “Bayesian image deconvolution anddenoising using complex wavelets,” presented at the IEEE Int. Conf.Image Process., Thessaloniki, Greece, 2001.

[41] V. Katkovnik, K. Egiazarian, and J. Astola, “A spatially adaptive non-parametric regression image deblurring,” IEEE Trans. Image Process.,vol. 14, no. 10, pp. 1469–1478, Oct. 2005.

[42] M. A. T. Figueiredo and R. D. Nowak, “An EM algorithm for waveletbased image restoration,” IEEE Trans. Image Process., vol. 12, no. 8,pp. 906–916, Aug. 2003.

[43] K. Guo, W. Lim, D. Labate, G. Weiss, and E. Wilson, “Waveletswith composite dilations and their MRA properties,” Appl. Comput.Harmon. Anal., vol. 20m, pp. 231–249, 2006.

[44] G. R. Easley, V. Patel, and D. M. Healy, Jr., “A M-channel directionalfilter bank compatible with the contourlet and shearlet frequencytiling,” presented at the SPIE Wavelets XII, San Diego, CA, Aug.26–30, 2007.

[45] K. Guo and D. Labate, “Optimally sparse multidimensional representa-tion using shearlets,” SIAM J. Math. Anal., vol. 39, no. 1, pp. 298–318,2007.

[46] G. H. Golub, M. Heath, and G. Wahba, “Generalized cross-validationas a method for choosing a good ridge parameter,” Technometrics, vol.21, no. 2, pp. 215–223, 1979.

[47] D. L. Donoho and I. Johnstone, “Ideal spatial adaptation via waveletshrinkage,” Biometrika, vol. 81, pp. 425–455, Dec. 1994.

[48] N. Weyrich and G. T. Warhola, “Wavelet shrinkage and generalizedcross validation for image denoising,” IEEE Trans. Image Process., vol.7, no. 1, pp. 82–90, Jan. 1998.

[49] M. Jansen, M. Malfait, and A. Bultheel, “Generalized cross validationfor wavelet thresholding,” Signal Process., vol. 56, no. 1, pp. 33–44,Jan. 1997.

[50] M. Jansen and A. Bultheel, “Multiple wavelet threshold estimation bygeneralized cross validation for images with correlated noise,” IEEETrans. Image Process., vol. 8, no. 7, pp. 947–953, Jul. 1999.

[51] F. Luisier, T. Blu, and M. Unser, “A new SURE approach to imagedenoising: Interscale orthonormal wavelet thresholding,” IEEE Trans.Image Process., vol. 16, no. 3, pp. 593–606, Mar. 2007.

[52] A. Papoulis and S. U. Pillai, Probability, Random Variables and Sto-chastic Processes, 3rd ed. New York: McGraw-Hill, 2002.

[53] A. N. Tikhonov and V. Y. Arsenin, Solution of Ill-Posed Problems.New York: Wiley, 1977.

[54] A. D. Hillery and R. T. Chin, “Iterative Wiener filters for image restora-tion,” IEEE Trans. Signal Process., vol. 39, no. 8, pp. 1892–1899, Aug.1991.

[55] S. Ghael, A. M. Sayeed, and R. G. Baraniuk, “Improved wavelet de-noising via empirical Wiener filtering,” in Proc. SPIE, Wavelet Appli-cations in Signal and Image Processing V, Oct. 1997, vol. 3169, pp.389–399.

[56] M. Crouse, R. Nowak, and R. Baraniuk, “Wavelet-based statisticalsignal processing using hidden Markov models,” IEEE Trans. SignalProcess., vol. 46, no. 4, pp. 886–902, Apr. 1998.

Vishal M. Patel (M’01) received the B.S. degrees inelectrical engineering and applied mathematics (withhonors) and the M.S. degree in applied mathematicsfrom North Carolina State University, Raleigh,in 2004 and 2005, respectively. He is currentlypursuing the Ph.D. degree in electrical engineeringat the University of Maryland, College Park.

His research interests include compressed sensing,radar imaging, inverse problems, pattern recognition,and biometrics.

Mr. Patel is a member of Eta Kappa Nu, Pi MuEpsilon, Phi Beta Kappa, and a student member of SIAM.


Glenn R. Easley (M’08) received the B.S. (withhonors) and M.A. degrees in mathematics fromthe University of Maryland, College Park, in 1993and 1996, respectively, and the Ph.D. degree incomputational science and informatics from GeorgeMason University in 2000.

Since 2000, he has been with System PlanningCorporation working in signal and image processing.He has also been a Visiting Assistant Professor forthe Norbert Wiener Center, University of Maryland,College Park, since 2005. His research interests

include computational harmonic analysis, with special emphasis on wavelettheory and time-frequency analysis, synthetic aperture radar, deconvolution,and computer vision.

Dennis M. Healy, Jr. received the B.S. degrees inphysics and mathematics and the Ph.D. degree inmathematics from the University of California atSan Diego (UCSD), La Jolla, in 1980 and 1986,respectively.

He is a Professor of mathematics at the Universityof Maryland, College Park, as well as programmanager for the Microsystems Technology Office atDARPA. Previously, he served as program managerfor DARPA’s Applied and Computational Mathe-matics Program in the Defense Sciences Office and

as Associate Processor at Dartmouth College with joint appointments in theDepartments of Mathematics and Computer Science. His research concernsapplied computational mathematics in real-world settings including medicalimaging, optical fiber communications, design and control of integratedsensor/processor systems, control of quantum systems, statistical patternrecognition, and fast nonabelian algorithms for data analysis.

ieee transactions on image processing, vol. …pvishalm/journal_pub/sheardec.pdf · ieee...

Documents