new method for performance evaluation of grayscale image denoising filters-ykg

7/31/2019 New Method for Performance Evaluation of Grayscale Image Denoising Filters-yKG

1/4

IEEE SIGNAL PROCESSING LETTERS, VOL. 17, NO. 5, MAY 2010 417

New Method for Performance Evaluationof Grayscale Image Denoising Filters

Fabrizio Russo, Senior Member, IEEE

AbstractCombining noise removal and detail preservation is achallenging issue in the design of image denoising filters. In thisletter a new full-reference method for objective evaluation of suchfeatures in grayscale pictures is presented. The proposed approachis based on the classification of the filtering errors in two classesthat respectively take into account the amount of noise still presentafter processing and the actual distortion produced by the filter.Results of computer simulations show the advantages of the pro-posed method with respect to other measures of filtering perfor-mance that are available in the literature.

Index TermsFilters, Gaussian noise, image analysis, image de-noising, impulse noise.

I. INTRODUCTION

DATA denoising is one of most important and activeresearch areas in digital image processing. Researchers

working in this field know very well that reducing noise withoutproducing image distortion is a very difficult and challengingtask. Indeed, noise removal and detail preservation are twoconflicting goals that must carefully be taken into account inthe design of any new filter. Thus, the availability of a standardtool for measuring these key effects would represent a usefulresource. The most commonly used approach to performanceevaluation of image denoising techniques typically combinesvisual inspection and objective measurements based on thecomputation of pixelwise differences between the original andthe processed image, such as mean squared error (MSE) andpeak signal-to-noise ratio (PSNR). Since MSE and PSNR bythemselves cannot characterize the behavior of a filter withrespect to noise cancellation and detail preservation, a visualanalysis of the filtered pictures is often reported in researchpapers to highlight these important features [1][3]. Differentapproaches to the evaluation of these effects in grayscale pic-tures have been proposed also. The degradation caused by thedenoising process is visually and numerically appraised in [4]by considering the method noise that is the effect that a filterwould produce on the original noise-free image. The powerspectrum of the difference between the denoised image andthe uncorrupted one is used in [5] to show how the denoisingtechnique can preserve the significant image features. Another

Manuscriptreceived December 10, 2009; revised January 22, 2010. Firstpub-lished February 05, 2010; current version published March 12, 2010. The as-sociate editor coordinating the review of this manuscript and approving it forpublication was Dr. Alexander Loui.

The author is with the Dipartimento di Elettrotecnica, Elettronica e Infor-matica, University of Trieste, I-34127 Trieste, Italy (e-mail: [email protected]).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/LSP.2010.2042516

interesting method is proposed in [6], where the correlationbetween the highpass filtered versions of the original noise-freeimage and the denoised picture is computed in order to evaluatethe edge preservation. Performance measures dealing withimpulse noise can exploit the fact that only some pixels arereplaced by noise pulses, whereas the other pixels remain attheir original values in the corrupted image [7], so the effective-ness of the impulse detection process can be analyzed [8], [9].Other approaches have also been investigated. As an example,the mean absolute error (MAE) is used in [10] to focus on thedetail preservation. Image quality metrics that, unlike MSE and

PSNR, aim at mimicking the human perception, have rapidlyemerged [11]. Some interesting examples are the universalquality index (UQI) [12], the structural similarity index (SSIM)[13], the M-SVD technique [14] and the recently introduced

Harris response-based image quality metric (HRQM) [15].In any case, visual inspection of the results is needed when ascalarindex is adopted for performance evaluation of denoisingfilters.

The goal ofVector approaches such as the vector root meansquared error (VRMSE) [16] is to perform a separate evaluationof the aforementioned effects. In this method the MSE is splitinto two components that respectively deal with noise cancella-tion and detail preservation. In the case of impulse noise, such

components are evaluated in the set of pixel coordinates corre-sponding to corrupted and uncorrupted pixels, respectively. Inthe case of Gaussian noise, these components take into accountthe uniform and nonuniform regions of the image, according toa map of edge gradients. Even if a vector method cannot com-pletely replace the visual inspection, it can extract useful infor-mation about key aspects of the denoising process.

In this letter, a new method for MSE decomposition is pre-sented. In addition to the general benefits of a vector approach,the proposed method yields the following advantages with re-spect to our previous technique: the definition of the MSE com-ponents does not depend upon the kind of noise distribution,the creation of an edge map is not required, the sensitivity to the

image distortion is better.The rest of this letter is organized as follows. Section II high-

lights the limitations of scalar indexes, Section III describes theproposed method, Section IV discusses the results of some com-puter simulations and, finally, Section V reports the conclusions.

II. LIMITATIONS OF SCALAR APPROACHES

As aforementioned, PSNR by itself cannot characterize thebehavior of a filter from the point of view of detail preservationand noise removal because different combinations of image dis-tortion and residual noise can yield the same result. A similarsituation can occur when scalar indexes that follow human per-

ception are adopted for this specific purpose. Indeed, different

1070-9908/$26.00 2010 IEEE

AlultIXDoM1a1UfIX Ra


2/4

418 IEEE SIGNAL PROCESSING LETTERS, VOL. 17, NO. 5, MAY 2010

Fig. 1. Portions of filtered pictures showing different combinations of distor-tion and residual noise but having the same MSSIM (a), (b) and the same IQM(c), (d) with respect to the original image..

TABLE IDIFFERENT KINDS OF NOISE CORRECTION WHEN

mixtures of residual noise and detail blur can produce the sameloss of perceived image quality and thus lead to the same score.

As a first example, we chose the MSSIM because it is a widelyadopted technique for image quality assessment and easy toimplement [13]. Portions of two processed pictures having thesame are shown in Fig. 1. In this example,we corrupted the 512 512 version of the Lena image by su-perimposing salt and pepper impulse noise with probability21%. We obtained the images in Fig. 1(a) and (b) by applyingmedian filters with different window sizes. The behavior of suchoperators is well known, so it is not a surprise that the 3 3 me-

dian [Fig. 1(a)] gives less noise removal and less distortion withrespect to the 5 5 operator [Fig. 1(b)]. However, this is notrevealed by the same MSSIM.

As a second example, we chose the new image quality metric(IQM) that is the core of the method described in [15]. Portionsof filtered images having the same are shownin Fig. 1(c) and (d). To obtain these results we corrupted theLena image by adding zero-mean Gaussian noise with stan-dard deviation and we applied arithmetic mean fil-ters with different window sizes because their behavior is wellknown. Clearly, the 5 5 filter is more effective than the 3 3operator in removing noise at the price of a worse detail blur, butthese different effects are not apparent from the IQM evaluation.

Actually, the same scores would mean that the filtered imageshave the same quality for a human observer. However, this is notsufficient for appraising the key features of a denoising filter. Aspecific tool is needed.

III. PROPOSED METHOD

The new method adopts a vector approach based on the clas-sification of the filtering errors in two classes, as follows: errorscaused by insufficient filtering that does not completely removethe noise (residual noise) and errors caused by excessive (orwrong) filtering that produces collateral distortion. Formally,suppose we deal with digitized imageshaving L gray levels (typ-ically ). Let be the pixel luminance at location

in the reference (noise-free) image. Let andbe the pixel luminances at the same location in the noisy

image and in the filtered one, respectively. We shall define thementioned error classes by means of the following characteristicfunctions:

(1)

(2)

where addresses the case of residual noise (RN) due toinsufficient filtering and deals with the collateral distor-tion (CD) caused by excessive (or wrong) smoothing. Indeed,let us suppose we have . The possible noise correc-tions provided by the filters output are listed in Table I.We can easily see that when the noise correction atlocation is none or too small, so some residualnoise is present after filtering. On the other hand,when the noise correction is wrong or excessive thus pro-

ducing collateral distortion.Similar observations can be done when . Hence,

a new definition of VRMSE (called type 2-VRMSE) is hereproposed as follows:

(3)

(4)

(5)

where and N is the total number of

processed pixels. Clearly we have:.

IV. RESULTS OF COMPUTER SIMULATIONS AND DISCUSSIONS

As discussed in Section II, classical and advanced scalar met-rics are not conceived to distinguish different mixtures of re-maining noise and unwanted distortion given by a filter. Vectormetrics can do it. Thus, in order to assess the performance of theproposed method, we considered for a comparison the recentlyintroduced vector technique [16] (here called type 1-VRMSE).The components and of type 1-VRMSEmea-sure the noise cancellation and the detail preservation as fol-lows. Such components respectively deal with the error terms

and , where. In the case of Gaussian noise, the function

is obtained from a map of edge gradients of the reference image.This function ranges from zero (sharp object contour) to unity(perfectly uniform neighborhood). In the case of impulse noise,

is a binary function such that if (noise pulse) and if (uncorruptedpixel).

In the first experiment we adopted the 512 512 Boatsimage corrupted by Gaussian noise and bilateralfiltering as denoising method [17]. We searched for differentsets of parameter values that give filtered images with the sameMSE. We chose a 9 9 window and the following param-

eter settings: , (better noise cancellation),and , (better detail preservation). The results



3/4

RUSSO: EVALUATION OF GRAYSCALE IMAGE DENOISING FILTERS 419

Fig. 2. (a) Type 1 and (b) type 2 vector errors for different filtering settings.

TABLE IIRMSE VALUES (BARBARA IMAGE)

TABLE IIIRMSE VALUES (LENA IMAGE)

TABLE IVRMSE VALUES (BOATS IMAGE)

of type 1 and type 2 VRMSE evaluations are graphically de-picted in Fig. 2. It can be seen that type 2-VRMSE is muchbetter at measuring the filtering differences. Clearly all vec-tors have the same length because they denote the same RMSEvalue. The angle between type 2 vectors, however, is much largerwhereas type 1 vectors are close to each other. The better sen-

sitivity of the proposed method to the image preservation wasconfirmed by a more extensive series of tests. In this secondexperiment we considered the test pictures Barbara, Lenaand Boats corrupted by Gaussian noise with standard devia-tion ranging from 10 to 25 and the results given by the pow-erful denoising method based on a local Gaussian scale mixture(LGSM) model [18], [19]. The type 1 and type 2 VRMSE eval-uations are listed in Tables IIIV. It can be observed that type2-VRMSE is better than type 1-VRMSE at extracting from theMSE the error component representing the image distortion. In-deed, the values of (fifth column) are greater than thecorresponding values of (third column) in all cases.

This behavior is further highlighted in a third experiment

where we generated filtered images that have almost the sameMSE and are hardly distinguishable from visual inspection. We

Fig. 3. Results given bythe LGSM (a)and NLM(b) filters; correspondingerrormaps showing the different contributions of RN (blue) and CD (red).

Fig. 4. Type 1 (a) and type 2 (b) vector errors for LGSM and NLM filters.

considered the Barbara image corrupted by Gaussian noisewith and the results given by two excellent filters suchas the previously mentioned technique [18] [Fig. 3(a)] and thenon local mean (NLM) filter [20] [Fig. 3(b)]. The results oftype1 and type 2 VRMSE evaluations are shown in Fig. 4. It can beseen that type 1 VRMSE vectors are poorly distinguishable (andso the filtering features), whereas type 2 VRMSE vectors arewell separate. Graphical representations of the squared errors

(blue) and (red) are also reportedin Fig. 3(c) (LGSM) and 3(d) (NLM). These error maps showthe different distributions of RN and CD that lead to the resultsin Fig. 4(b). In a fourth experiment, we considered the fuzzyrank LUM (FLUM) smoother and its behavior in the removal

of impulse noise [10]. The FLUM smoother is an elegant andeffective method that can have better detail preservation thanthe LUM smoother while achieving similar noise cancellation.This behavior is very clearly shown in Fig. 5, where the inputpicture is the House image corrupted by impulse noise withprobability 10%. In this case the results given by type 1 and type2 VRMSE are similar. However, it should be observed that thedefinition oftype 1 vector components for impulse noise is dif-ferent from that for Gaussian noise. Conversely, the definitionoftype 2 vector components does not change.

In a fifth experiment, we considered the Lena image cor-rupted by impulse noise with probability p ranging from 5%to 25% and the results given by the 3 3 median. The corre-

sponding and values are listed in Table V.The classical RMSE and MAE evaluations are also reported for



4/4

420 IEEE SIGNAL PROCESSING LETTERS, VOL. 17, NO. 5, MAY 2010

Fig. 5. Results of (a) FLUM and (b) LUM filters; (c) corresponding errors.

TABLE VRMSE AND MAE VALUES (LENA IMAGE)

TABLE VIRMSE VALUES FOR THE IMAGES IN FIG. 1

a comparison. The proposed approach offers two key advan-tages over them.

1) The new method gives more accurate global measures of

noise cancellation and detail preservation because it cantotally separate these effects, whereas RMSE and MAEcannot. The MAE is more sensitive to distortion than theMSE. However, it depends upon the remaining noise too.The actual influence of this component can be revealed ifwe apply our decomposition scheme to the MAE (sixth andseventh column of Table V).

2) The error classification defined by (1) and (2) permits usto know where residual noise and distortion occur, as inFig. 3(c) and (d).

Finally, we list in Table VI the RMSE values for the imagesin Fig. 1.

V. CONCLUSION

In this letter, a new approach to performance evaluation ofgrayscale image denoising filters has been presented. The pro-posed method is conceptually simple and, unlike other tech-niques, focuses on the actual filtering action. Indeed, for eachimage pixel, the correctness of the noise correction is evalu-ated and this information is used to decompose the MSE intotwo components that measure the residual noise and the collat-

eral distortion. Although the proposed approach cannot totallyreplace visual inspection because it does not consider humanperception, the new vector method overcomes the limitations ofcurrent scalar indexes for this application. It also offers signifi-cant advantages over our previous vector technique. The methodis computationally light too (it requires twice the execution timeof a classical MSE evaluation).

REFERENCES

[1] M. Mahmoudi andG. Sapiro, Fastimageand video denoisingvia non-localmeans of similar neighborhoods,IEEE Signal Process. Lett., vol.12, no. 12, pp. 839842, Dec. 2005.

[2] D. K. Hammond and E. P. Simoncelli, Image modeling and denoisingwith orientation-adapted Gaussian scale mixtures, IEEE Trans. ImageProcess., vol. 17, no. 11, Nov. 2008.

[3] H. Ibrahim, N. S. P. Kong, andT. F. Ng,Simple adaptive medianfilterfor the removal of impulse noise from highly corrupted images, IEEETrans. Consumer Electron., vol. 54, no. 4, pp. 19201927, 2008.

[4] A. Buades, B. Coll, and J. M. Morel, A review of image denoisingalgorithms, with a new one, Multiscale Model. Simul., vol. 4, no. 2,pp. 490530, 2005.

[5] S. P. Awate and R. T. Whitaker, Feature-preserving MRI denoising:A nonparametric empirical Bayes approach, IEEE Trans. Med. Imag.,vol. 26, no. 9, pp. 12421255, Sep. 2007.

[6] F. Sattar, L. Floreby, G. Salomonsson, and B. Loevstroem, Imageenhancement based on a nonlinear multiscale method, IEEE Trans.

Image Process., vol. 6, no. 6, pp. 888895, Jun. 1997.[7] F. Russo, Nonlinear filters based on fuzzy models, in Nonlinear

Image Processing, S. K. Mitra and G. L. Sicuranza, Eds. San Diego,CA: Academic, 2001, pp. 355374.

[8] I. Aizenberg, C. Butakoff, and D. Paliy, Impulsive noise removalusingthreshold boolean filtering based on the impulse detecting functions,

IEEE Signal Process. Lett., vol. 12, no. 1, pp. 6366, Jan. 2005.[9] I. Frosio and N. A. Borghese, Statistical based impulsive noise re-

moval in digital radiography, IEEE Trans. Med. Imag., vol. 28, no. 1,pp. 316, Jan. 2009.

[10] Y. Nieand K. E. Barner, Fuzzyrank LUMfilters,IEEE Trans. ImageProcess., vol. 15, no. 12, pp. 36363654, Dec. 2006.

[11] H. R. Sheikh, M. F. Sabir, and A. C. Bovik, A statistical evaluationof recent full reference image quality assessment algorithms, IEEETrans. Image Process., vol. 15, no. 11, pp. 34413451, Nov. 2006.

[12] Z. Wang and A. C. Bovik, A universal image quality index, IEEESignal Process. Lett., vol. 9, no. 3, pp. 8184, Mar. 2002.

[13] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, Imagequality assessment: From error visibility to structural similarity, IEEETrans. Image Process., vol. 13, no. 4, pp. 600612, Apr. 2004.

[14] A. Shnayderman, A. Gusev, and A. M. Eskicioglu, An SVD-basedgrayscale image quality measure for local and global assessment,

IEEE Trans. Image Process., vol. 15, no. 2, pp. 422429, Feb. 2006.[15] D.-O. Kim and R.-H. Park, New image quality metric using the Harris

response, IEEE Signal Process. Lett., vol. 16, no. 7, pp. 616619, Jul.2009.

[16] A. De Angelis, A. Moschitta, F. Russo, and P. Carbone, A vector ap-proach for image quality assessment and some metrological considera-

tions,IEEE Trans. Instrum. Meas, vol. 58,no. 1, pp. 1425, Jan. 2009.[17] C. Tomasi and R. Manduchi, Bilateral filtering for gray and color im-ages, in Proc. 6th Int. Conf. Computer Vision, Bombay, India, 1998,pp. 839846.

[18] J. Portilla, V. Strela, M. Wainwright, and E. P. Simoncelli, Imagedenoising using scale mixtures of Gaussians in the wavelet domain,

IEEE Trans. Image Process., vol. 12, no. 11, pp. 13381351, Nov.2003.

[19] [Online]. Available: http://decsai.ugr.es/~javier/denoise/test_images/index.htm

[20] A. Buades, B. Coll, and J. M. Morel, A non-local algorithm for imagedenoising, in Proc. IEEE Int. Conf. on Computer Vision and Pattern

Recognition, 2005, vol. 2, pp. 6065.


new method for performance evaluation of grayscale image denoising filters-ykg

Documents