research article single image super-resolution via image...

Research ArticleSingle Image Super-Resolution via 119871

0Image Smoothing

Zhang Liu12 Qi Huang1 Jian Li1 and Qi Wang13

1School of Energy Science and Engineering University of Electronic Science and Technology of China Chengdu 611731 China2College of Physics and Engineering Chengdu Normal University Chengdu 611130 China3Electrical and Electronic Engineering Department Chengdu University of Technology Chengdu 611730 China

Correspondence should be addressed to Zhang Liu liuzhangha110163com

Received 19 October 2014 Accepted 4 December 2014 Published 29 December 2014

Academic Editor Jose R C Piqueira

Copyright copy 2014 Zhang Liu et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

We propose a single image super-resolution method based on a 1198710smoothing approach We consider a low-resolution image as

two parts one is the smooth image generated by the 1198710smoothing method and the other is the error image between the low-

resolution image and the smoothing image We get an intermediate high-resolution image via a classical interpolation and thengenerate a high-resolution smoothing image with sharp edges by the 119871

0smoothing method For the error image a learning-based

super-resolution approach keeping image details well is employed to obtain a high-resolution error image The resulting high-resolution image is the sum of the high-resolution smoothing image and the high-resolution error image Experimental resultsshow the effectiveness of the proposed method

1 Introduction

Image super-resolution is an active field recently in imageprocessing It tends to obtain the high-resolution image viaone or multiple low-resolution images If we get the high-resolution image only by one low-resolution input we call itsingle image super-resolution The other case is called clas-sical image super-resolution Obviously single image super-resolution is more useful and challenging for the practicalapplications since only one low-resolution image is availablesometimes due to the limitation of hardware or other reasonsIn particular we address single image super-resolution inthe paper Image super-resolution has many applicationsfor example medical imaging magnetic resonance imaging(MRI) synthetic aperture radar (SAR) and high definitiontelevision How to develop an effective super-resolutionmodel and algorithm is very important

There are many image super-resolution methods up tonow They can be classified into three categories interpo-lation-based methods reconstruction-based methods andlearning-based methods Interpolation-based methods tendto interpolate the unknown points via the known neighborpoints The two most famous interpolation methods arenearest-neighbor interpolation and bicubic interpolation Inaddition more interpolation methods can be found in [1ndash3]

Interpolation-based methods are very popular and fast butthey have to face some drawbacks for example jaggy effector blur effect on image edges Many reconstruction methods[4ndash10] have been proposed They obtain the resulting super-resolution images via some reconstruction ideas In [9] Shanet al propose a fast imagevideo super-resolution approachby minimizing an energy function This method can usefast Fourier transformation (FFT) algorithm to accelerate thespeed It upsamples a low-resolution image and then appliesa fast reconstruction method to the upsampled image to getthe resulting high-resolution output In [10] Chambolle andPock propose a first-order primal-dual algorithm for convexmodels and apply this algorithm to single image super-resolu-tion In addition authors also give the convergence analysisof corresponding algorithm Learning-based methods are avery popular tool to address image super-resolution problems[11ndash24] Excellent results can be obtained via learning-basedmethods However before implementing the method wehave to generate two training data sets One data set is formedby a mass of low-resolution patches the other is formedby corresponding high-resolution patches After getting therelation between the two training data sets we apply it toa given low-resolution image to get high-resolution imageObviously this approach depends on the selection of trainingdata sets and leads to expensive computation In [19 20]

Hindawi Publishing CorporationMathematical Problems in EngineeringVolume 2014 Article ID 974509 8 pageshttpdxdoiorg1011552014974509

2 Mathematical Problems in Engineering

Yang et al propose a sparse signal representation methodfor single image super-resolution They search a sparse rep-resentation for each patches of the low-resolution image andcompute the coefficients and then apply the computed coef-ficients to generate the high-resolution image Based on thework of [19 20] Zeyde et al [22] simplify the overall processand reduce the computation complexity of algorithm In [23]Timofte et al propose a fast image super-resolution methodwhich makes no compromise on image quality It proposesan anchored neighborhood regression and utilizes globalcollaborative coding for speedup In [24] Dong et al learn amapping between low-resolution and high-resolution imagesand then consider themapping as a deep convolutional neuralnetwork (CNN) The CNN takes the low-resolution image asinput and outputs the resulting high-resolution image

In addition some different methods also have beenproposed for example frequency technique [25] pixel clas-sification method [26] and others [10 27ndash29]

In this paper we divide a low-resolution image into twoparts one part is smoothing image with sharp edges which isgenerated by 119871

0smoothing method the other part is error

image which is the difference between the low-resolutionimage and the smoothing image We get the high-resolutionsmoothing image by applying 119871

0smoothing method to

an intermediate high-resolution image The high-resolutionsmoothing image contains very important features of super-resolution images sharp edges For the error image weupsample it using an approach of keeping image details toget high-resolution error image The final resulting super-resolution image is the sum of high-resolution smoothingimage and high-resolution error image

The organization of this paper is as follows In Section 2we introduce an important image smoothing method whichis utilized in our work In addition we give the proposedmethod which uses the smoothingmethod and one learning-based method The experimental results are shown in Sec-tion 3 Finally we give the conclusions in Section 4

2 The Proposed Method and Related Work

21 1198710Image Smoothing In [30] Xu et al proposed a

novel image smoothingmethod via1198710gradientminimization

which is used to control the number of nonzero gradients Bycontrolling the number of nonzero gradients they establisha 1198710model to smooth the high-frequency details of image

preserving prominent image structures

1D Signal Smoothing For 1D case 119862(119891) counts the number ofneighbor pixels 119891

119901and 119891

119901+1

119862 (119891) = 119901 | 10038161003816100381610038161003816119891119901 minus 119891119901+110038161003816100381610038161003816= 0 (1)

where 119891119901is the smoothing result and represents the count-

ing operator and outputs the number of 119901 satisfying |119891119901minus

119891119901+1| = 0 Note that it can give one value to 119862(119891) to control

the smoothness of image Although 119862(119891) is not functionalthe specific objective function can be given as follows

min119891

sum

119901

(119891119901minus 119892119901)2

st 119862 (119891) = 119896 (2)

+ =

LR E

H S HR

Method 1 Method 2

=+

H0

L0

L0 smooth

Figure 1 Illustration of the proposed method

where 119896 isin N is the given value and 119892119901is the known

discrete signal at location 119901 This optimization model is verypowerful to abstract the image structure It flattens imagedetails effectively and preserves sharp image edges obviouslyThe property of preserving sharp edges is the most importantmotivation of this paper to deal with image super-resolution

Actually 119896 can be set from zero to thousands accordingto practical case A general regularity form is employed tobalance the structure flattening term and the fidelity term itis shown as follows

min119891

sum

119901

(119891119901minus 119892119901)2

+ 120582 sdot 119862 (119891) (3)

where 120582 is the regularity parameter which can control thesignificance of 119862(119891) The parameter also can be viewed as asmoothing factor a larger 120582 will get fewer image edges Inparticular the number of nonzero gradients 119896 is monotonewith respect to 1120582

2D Image Smoothing For 2D image case 119868 is denoted asthe input image and 119878 is denoted as the resulting smoothingimage The gradient of image at pixel 119901 is represented bynabla119878119901= (120597119909119878119901 120597119910119878119901)119879 Similar to (1) it can get the number of

nonzero gradients in the following formula119862 (119878) = 119901 | 10038161003816100381610038161003816120597119909119878119901

10038161003816100381610038161003816+10038161003816100381610038161003816120597119910119878119901

10038161003816100381610038161003816= 0 (4)

Similar to (3) we get the image smoothing modelaccording to (4)

min119878

sum

119901

(119878119901minus 119868119901)2

+ 120582 sdot 119862 (119878) (5)

where 120582 is the balance parameter similar to (3) Actually theterm119862(119878) smoothes the image details and (119878

119901minus119868119901)2 keeps the

main image structure similarly The corresponding solver of(5) will be given in the following

Solving For the optimization problem (5) it is a nonde-terministic polynomial-time (NP) hard problem to find theglobal optimal solution Motivated by Wang et al [32]a splitting scheme is proposed to solve the optimizationproblem Two auxiliary variables ℎ

119901and V119901are introduced to

substitute 120597119909119878119901and 120597119910119878119901 Equation (5) can be rewritten as

min119878ℎVsum

119901

(119878119901minus 119868119901)2

+ 120582 sdot 119862 (ℎ V)

+ 120573 ((120597119909119878119901minus ℎ)2

+ (120597119910119878119901minus V)2

)

(6)

Mathematical Problems in Engineering 3

(a) (b) (c)

Figure 2 (a) High-resolution image (b) high-resolution smoothing image by 1198710smoothing method (c) high-resolution error image by the

learning-based method [31] (for better vision adding 03 to the error image)

Input Image 119868 smoothing weight 120582 parameter 1205730 120573max rate 120581

(1) Initialization 119878 larr 119868 120573 larr 1205730 119894 larr 0

(2)While 120573 lt 120573max(3) Solving ℎ(119894)

119901 119907(119894)

119901via (10) when fixing 119878(119894)

(4) Solving 119878(119894+1) via (8) when fixing ℎ(119894)119901 119907(119894)

119901

(5) Update 120573 larr 120581120573(6) 119894 larr 119894 + 1

(7) endOutput Smoothing image 119878

Algorithm 1 Image smoothing via 1198710gradient minimization

where 120573 is an adaptive parameter to control the distancebetween (ℎ V) and their gradients (120597

119909119878119901 120597119910119878119901) For solving

one variable it should fix the other variable obtained fromthe previous iteration This process can be depicted as twosubproblems

S-Subproblem We get 119878 by solving the following minimiza-tion problem

min119878

sum

119901

(119878119901minus 119868119901)2

+ 120573 ((120597119909119878119901minus ℎ)2

+ (120597119910119878119901minus V)2

) (7)

The problem has global minimum solution due toquadratic function It may be speed up by following fastFourier transform (FFT) (see details in [32]) Consider

119878 = Fminus1

times (

I + 120573 (F (120597119909)lowastF (ℎ) +F (120597119910)

lowast

F (V))

F (1) + 120573 (F (120597119909)lowastF (120597119909) +F (120597

119910)lowast

F (120597119910))

)

(8)

where F is FFT operator F(1) is the Fourier transform ofdelta function and Flowast is the complex conjugate Note that119878 is computed very fast due to the FFT operator which hascomplexity O(119873 log(119873))

(ℎ V)-Subproblem For (ℎ V)-subproblem the correspondingminimization problem is as follows

minℎVsum

119901

120573 ((120597119909119878119901minus ℎ)2

+ (120597119910119878119901minus V)2

) + 120582 sdot 119862 (ℎ V) (9)

where 119862(ℎ V) = sum119901119867(|ℎ119901| + |V119901|)119867(|ℎ

119901| + |V119901|) = 1 if |ℎ

119901| +

|V119901| = 0 and 119867(|ℎ

119901| + |V119901|) = 0 otherwise Thus (9) can be

rewritten as

sum

119901

minℎ119901 V119901120573 ((120597119909119878119901minus ℎ)2

+ (120597119910119878119901minus V)2

) + 120582 sdot 119867 (10038161003816100381610038161003816ℎ119901

10038161003816100381610038161003816+10038161003816100381610038161003816V119901

10038161003816100381610038161003816)

(10)

In [30] it has been proved that for each pixel 119901 119864119901=

120573((120597119909119878119901minus ℎ)2+ (120597119910119878119901minus V)2) + 120582 sdot 119867(|ℎ

119901| + |V119901|) reaches its

minimum under the following condition

(ℎ119901 V119901) =

(0 0) (120597119909119878119901)2

+ (120597119910119878119901)2

le120582

120573

(120597119909119878119901 120597119910119878119901) otherwise

(11)

The resulting algorithm for image smoothing is as shownin Algorithm 1

Step (5) is to update 120573 adaptively via a parameter 120581 foreach iteration The resulting 119878 is the smoothing image whichflattens image details but preserves sharp edges


(a) LR (b) NN (c) Bicubic (d) Ours

Figure 3 (a) Low-resolution images (b) high-resolution images by nearest-neighbor interpolation (c) high-resolution images by bicubicinterpolation (d) high-resolution images by the proposed method The upscaling factors of first to third row are 3 the last row is 2


(a) LR (b) Bicubic (d) Ours(c) 07rsquoTIP

Figure 4 (a) Low-resolution images (b) high-resolution images by bicubic interpolation (c) high-resolution error image by the learning-based method [31] (d) high-resolution images by the proposed method The upscaling factors are 4

22The CombinedMethod for Image Super-Resolution Sharpedges are very important features to super-resolution imagesIn our work we utilize the property of preserving sharpedges of 119871

0image smoothing to enhance the quality of super-

resolution image We divide one low-resolution image intotwo parts (1) smoothing image obtained by 119871

0smoothing

method and (2) error image obtained by the difference of thelow-resolution image and the smoothing image After thisprocess we take two steps to realize the proposed methodFirst we upsample the low-resolution image to get intermedi-ate high-resolution image by a fast and classical interpolationmethod and then apply 119871

0smoothing method to it to

get high-resolution smoothing image Note that the high-resolution smoothing image has sharp edges Second theerror image will be upsampled to high-resolution error imagevia one learning-based super-resolution method of keepingimage details well

In Figure 1 we show the proposed strategy for imagesuper-resolutionThis strategy combines twomethods of dif-ferent image applications for image super-resolution In par-ticular ldquoLRrdquo represents the low-resolution image ldquo119871

0rdquo is the

low-resolution smoothing image via 1198710smoothing method

ldquo119864rdquo is the error image between ldquoLRrdquo and ldquo1198710rdquo Actually

ldquo1198710rdquo is just to get the low-resolution error image 119864 ldquo119867

0rdquo is

the upsampled image by one classical interpolation method(here we use ldquobicubicrdquo interpolation method as ldquoMethod 1rdquo)ldquo119867rdquo is the high-resolution smoothing image by the same 119871

0

smoothing method ldquo119878rdquo is the high-resolution error image by

one learning-basedmethod which can preserve image detailswell (in our work we use one learning-based method [31] asldquoMethod 2rdquo)

We present Figure 2 to specify the rationality of theproposed work From Figure 2 the resulting high-resolutionimage (Figure 2(a)) is the sum of high-resolution smoothingimage (Figure 2(b)) and high-resolution error image (Fig-ure 2(c)) Note that the high-resolution smoothing imagehas sharp image edges and the high-resolution error imagecontains obvious image details It demonstrates that ourproposed work can combine the advantages of 119871

0image

smoothing (preserving sharp edges) and the detail-preservedmethod [31] (keeping obvious image details)

3 Results

In this section we compare the proposed method with somecompetitive image super-resolution methods for example01rsquoTIP [1] 06rsquoTIP [3] and 07rsquoTIP [8] The experimentalcomputer is a laptop with 325Gb RAM and Intel Core i3-2370M CPU 240GHz All experiments are implementedon MATLAB (R2010a) For the corresponding parameterswe set 120581 = 2 120582 = 0005 120573max = 10

5 and 1205730= 2120582

In addition the images with different types and upscalingfactors are employed to test the effectiveness of the comparedmethods

In Figure 3 we compare our method with two classicalinterpolation methods (nearest-neighbor interpolation and


(c) 01rsquoTIP

(c) 01rsquoTIP

(d)06rsquoTIP

(d)06rsquoTIP

(e) 07rsquoTIP

(e) 07rsquoTIP

(a) LR

(a) LR

(b) Bicubic

(b) Bicubic

(f) Ours

(f) Ours

Figure 5 (a) Low-resolution images (b) high-resolution images by bicubic interpolation (c) high-resolution images by 01rsquoTIP [1] (d) high-resolution images by 06rsquoTIP [3] (e) high-resolution images by [8] (f) high-resolution images by the proposed methodThe upscaling factorsare 2


bicubic interpolation) Different images for example butter-fly andflower are employed for experiments From the figureit is easy to know that the resulting super-resolution imagesby the proposed method keep sharper image edges thannearest-neighbor and bicubic interpolations In particularthe resulting super-resolution images by nearest-neighborinterpolation show jaggy effect and the super-resolutionimages by bicubic interpolation show blur effect

In Figure 4 a competitive kernel regression method [8] isemployed to compare with the proposed method We knowthat the resulting images by ourmethod preserve sharp edgesbetter than other methods The kernel regression method [8]and bicubic interpolation all show blur effect on the edges

From Figure 5 more competitive super-resolution meth-ods are compared with our method We employ two imageswith different types in the experiments one type is naturalimage and the other type is comic image From the figurebicubic interpolation 01rsquoTIP [1] 06rsquoTIP [3] and 07rsquoTIP [8]all show blur effect especially on image edges The proposedmethod performs best not only on image edges but also onnonedge regions

4 Conclusions

In this paper a novel image super-resolution method wasproposed based on an image smoothing strategy We applied1198710image smoothing method to a given low-resolution

image to get its low-resolution smoothing image whichpreserved sharp edges Besides a low-resolution error imagewas obtained by the difference between the low-resolutionimage and the low-resolution smoothing image For thelow-resolution error image and the given low-resolutionimage we took two steps to generate the resulting super-resolution image First we upsampled the low-resolutionimage by bicubic interpolation to get intermediate high-resolution image and then applied the 119871

0smoothing method

to the intermediate high-resolution image to generate thehigh-resolution smoothing image Second a learning-basedsuper-resolution method was utilized to upsample the low-resolution error image to get high-resolution error imageThe learning-based method could preserve image detailswell The final super-resolution image was the sum of thehigh-resolution smoothing image and the high-resolutionerror image In experimental section we employed low-resolution images of different types to test our methodResults demonstrated that the proposed method performedbetter than some competitive image super-resolution meth-ods especially preserving sharp image edges

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgment

Theauthors thank the reviewers for their valuable comments

References

[1] X Li and M T Orchard ldquoNew edge-directed interpolationrdquoIEEE Transactions on Image Processing vol 10 no 10 pp 1521ndash1527 2001

[2] N Mueller Y Lu and M Do ldquoImage interpolation using mul-tiscale geometric representationsrdquo in Computational Imagingvol 6498 of Proceedings of SPIE 2007

[3] L Zhang andXWu ldquoAn edge-guided image interpolation algo-rithm via directional filtering and data fusionrdquo IEEE Transac-tions on Image Processing vol 15 no 8 pp 2226ndash2238 2006

[4] M Irani and S Peleg ldquoSuper resolution from image sequencesrdquoin Proceedings of the 10th International Conference on PatternRecognition (ICPR rsquo90) pp 115ndash120 June 1990

[5] M Irani and S Peleg ldquoMotion analysis for image enhancementresolution occlusion andtransparencyrdquo Journal of Visual Com-munication and Image Representation vol 4 no 4 pp 324ndash3351993

[6] T Komatsu T Igarashi K Aizawa and T Saito ldquoVery high res-olution imaging scheme with multiple different-aperture cam-erasrdquo Signal Processing Image Communication vol 5 no 5-6pp 511ndash526 1993

[7] P Chatterjee S Mukherjee S Chaudhuri and G SeetharamanldquoApplication of Papoulis-Gerchberg method in image super-resolution and inpaintingrdquo The Computer Journal vol 52 no1 pp 80ndash89 2009

[8] H Takeda S Farsiu and P Milanfar ldquoKernel regression forimage processing and reconstructionrdquo IEEE Transactions onImage Processing vol 16 no 2 pp 349ndash366 2007

[9] Q Shan Z Li J Jia and C Tang ldquoFast imagevideo upsam-plingrdquo ACM Transactions on Graphics vol 27 no 5 pp 1ndash72008

[10] A Chambolle and T Pock ldquoA first-order primal-dual algorithmfor convex problems with applications to imagingrdquo Journal ofMathematical Imaging and Vision vol 40 no 1 pp 120ndash1452011

[11] W T Freeman E C Pasztor and O T Carmichael ldquoLearninglow-level visionrdquo International Journal of Computer Vision vol40 no 1 pp 25ndash47 2000

[12] W T Freeman and E C Pasztor ldquoMarkov networks for super-resolutionrdquo in Proceedings of the 34th Annual Conference onInformation Sciences and Systems 2000

[13] E Gur and Z Zalevsky ldquoSingle-Image digital super-resolutiona revised Gerchberg-Papoulis algorithmrdquo IAENG InternationalJournal of Computer Science vol 34 no 2 pp 251ndash255 2007

[14] J Sun N N Zheng H Tao and H Shum ldquoImage hallucinationwith primal sketch priorsrdquo inProceedings of the IEEEConferenceon Computer Vision and Pattern Recognition (CVPRrsquo 03) vol 2pp 729ndash736 2003

[15] Y Zhao J Yang Q Zhang L Song Y Cheng and Q PanldquoHyperspectral imagery super-resolution by sparse representa-tion and spectral regularizationrdquo EURASIP Journal on Advancesin Signal Processing vol 2011 no 1 article 87 2011

[16] M F Tappen B C Russell and W T Freeman ldquoExploiting thesparse derivative prior for super-resolution and image demo-saicingrdquo in Proceedings of the IEEE Workshop on Statistical andComputational Theories of Vision 2003

[17] W T Freeman T R Jones and E C Pasztor ldquoExample-basedsuper-resolutionrdquo IEEE Computer Graphics and Applicationsvol 22 no 2 pp 56ndash65 2002


[18] J Sun Z Xu and H-Y Shum ldquoImage super-resolution usinggradient profile priorrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo08) pp 1ndash8 Anchorage Alaska USA June 2008

[19] J Yang JWright T Huang and YMa ldquoImage super-resolutionas sparse representation of raw image patchesrdquo in Proceedings ofthe 26th IEEE Conference on Computer Vision and PatternRecognition (CVPR rsquo08) pp 1ndash8 Anchorage Alaska USA June2008

[20] J Yang J Wright T S Huang and Y Ma ldquoImage super-resolution via sparse representationrdquo IEEE Transactions onImage Processing vol 19 no 11 pp 2861ndash2873 2010

[21] J Yang ZWang Z Lin S Cohen and T Huang ldquoCoupled dic-tionary training for image super-resolutionrdquo IEEE Transactionson Image Processing vol 21 no 8 pp 3467ndash3478 2012

[22] R Zeyde M Elad and M Protter ldquoOn single image scale-upusing sparse-representationsrdquo in Curves and Surfaces vol 6920of Lecture Notes in Computer Science pp 711ndash730 Springer2012

[23] R Timofte V Smet and L Gool ldquoAnchored neighborhoodregression for fast example-based super-resolutionrdquo in Proceed-ings of the International Conference on Computer Vision (ICCVrsquo13) December 2013

[24] C Dong C Loy K He and X Tang ldquoLearning a deep convolu-tional network for image super-resolutionrdquo inComputer Vision-ECCV 2014 vol 8692 of Lecture Notes in Computer Science pp184ndash199 Springer New York NY USA 2014

[25] S Borman and R L Stevenson ldquoSuper-resolution from imagesequences-a reviewrdquo in Proceedings of the Midwest Symposiumon Circuits and Systems pp 374ndash378 Notre Dame Ind USA1998

[26] C B Atkins C A Bouman and J P Allebach ldquoOptimal imagescaling using pixel classificationrdquo in Proceedings of the IEEEInternational Conference on Image Processing (ICIP rsquo01) pp864ndash867 Thessaloniki Greece October 2001

[27] E Candes and C Fernandez-Granda ldquoTowards a mathemati-cal theory of super-resolutionrdquo Communications on Pure andApplied Mathematics vol 67 no 6 pp 906ndash956 2014

[28] F Viola A W Fitzgibbon and R Cipolla ldquoA unifyingresolution-independent formulation for early visionrdquo in Pro-ceedings of the IEEE Conference on Computer Vision and PatternRecognition (CVPR rsquo12) pp 494ndash501 2012

[29] G Freedman and R Fattal ldquoImage and video upscaling fromlocal self-examplesrdquoACMTransactions on Graphics vol 30 no2 article 12 2011

[30] L Xu C Lu Y Xu and J Jia ldquoImage smoothing via 1198710gradient

minimizationrdquo ACM Transactions on Graphics vol 30 no 6article 174 2011

[31] K I Kim and Y Kwon ldquoSingle-image super-resolution usingsparse regression and natural image priorrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 32 no 6 pp1127ndash1133 2010

[32] YWang J YangW Yin and Y Zhang ldquoA new alternatingmin-imization algorithm for total variation image reconstructionrdquoSIAM Journal on Imaging Sciences vol 1 no 3 pp 248ndash2722008

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of


Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of


Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of


Mathematical PhysicsAdvances in

Complex AnalysisJournal of


OptimizationJournal of


CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of


Operations ResearchAdvances in

Journal of


Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences


The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014


Algebra

Discrete Dynamics in Nature and Society



Decision SciencesAdvances in

Discrete MathematicsJournal of


Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of


Yang et al propose a sparse signal representation methodfor single image super-resolution They search a sparse rep-resentation for each patches of the low-resolution image andcompute the coefficients and then apply the computed coef-ficients to generate the high-resolution image Based on thework of [19 20] Zeyde et al [22] simplify the overall processand reduce the computation complexity of algorithm In [23]Timofte et al propose a fast image super-resolution methodwhich makes no compromise on image quality It proposesan anchored neighborhood regression and utilizes globalcollaborative coding for speedup In [24] Dong et al learn amapping between low-resolution and high-resolution imagesand then consider themapping as a deep convolutional neuralnetwork (CNN) The CNN takes the low-resolution image asinput and outputs the resulting high-resolution image

In addition some different methods also have beenproposed for example frequency technique [25] pixel clas-sification method [26] and others [10 27ndash29]

In this paper we divide a low-resolution image into twoparts one part is smoothing image with sharp edges which isgenerated by 119871

0smoothing method the other part is error

image which is the difference between the low-resolutionimage and the smoothing image We get the high-resolutionsmoothing image by applying 119871

0smoothing method to

an intermediate high-resolution image The high-resolutionsmoothing image contains very important features of super-resolution images sharp edges For the error image weupsample it using an approach of keeping image details toget high-resolution error image The final resulting super-resolution image is the sum of high-resolution smoothingimage and high-resolution error image

The organization of this paper is as follows In Section 2we introduce an important image smoothing method whichis utilized in our work In addition we give the proposedmethod which uses the smoothingmethod and one learning-based method The experimental results are shown in Sec-tion 3 Finally we give the conclusions in Section 4

2 The Proposed Method and Related Work

21 1198710Image Smoothing In [30] Xu et al proposed a

novel image smoothingmethod via1198710gradientminimization

which is used to control the number of nonzero gradients Bycontrolling the number of nonzero gradients they establisha 1198710model to smooth the high-frequency details of image

preserving prominent image structures

1D Signal Smoothing For 1D case 119862(119891) counts the number ofneighbor pixels 119891

119901and 119891

119901+1

119862 (119891) = 119901 | 10038161003816100381610038161003816119891119901 minus 119891119901+110038161003816100381610038161003816= 0 (1)

where 119891119901is the smoothing result and represents the count-

ing operator and outputs the number of 119901 satisfying |119891119901minus

119891119901+1| = 0 Note that it can give one value to 119862(119891) to control

the smoothness of image Although 119862(119891) is not functionalthe specific objective function can be given as follows

min119891

sum

119901

(119891119901minus 119892119901)2

st 119862 (119891) = 119896 (2)

+ =

LR E

H S HR

Method 1 Method 2

=+

H0

L0

L0 smooth

Figure 1 Illustration of the proposed method

where 119896 isin N is the given value and 119892119901is the known

discrete signal at location 119901 This optimization model is verypowerful to abstract the image structure It flattens imagedetails effectively and preserves sharp image edges obviouslyThe property of preserving sharp edges is the most importantmotivation of this paper to deal with image super-resolution

Actually 119896 can be set from zero to thousands accordingto practical case A general regularity form is employed tobalance the structure flattening term and the fidelity term itis shown as follows

min119891

sum

119901

(119891119901minus 119892119901)2

+ 120582 sdot 119862 (119891) (3)

where 120582 is the regularity parameter which can control thesignificance of 119862(119891) The parameter also can be viewed as asmoothing factor a larger 120582 will get fewer image edges Inparticular the number of nonzero gradients 119896 is monotonewith respect to 1120582

2D Image Smoothing For 2D image case 119868 is denoted asthe input image and 119878 is denoted as the resulting smoothingimage The gradient of image at pixel 119901 is represented bynabla119878119901= (120597119909119878119901 120597119910119878119901)119879 Similar to (1) it can get the number of

nonzero gradients in the following formula119862 (119878) = 119901 | 10038161003816100381610038161003816120597119909119878119901

10038161003816100381610038161003816+10038161003816100381610038161003816120597119910119878119901

10038161003816100381610038161003816= 0 (4)

Similar to (3) we get the image smoothing modelaccording to (4)

min119878

sum

119901

(119878119901minus 119868119901)2

+ 120582 sdot 119862 (119878) (5)

where 120582 is the balance parameter similar to (3) Actually theterm119862(119878) smoothes the image details and (119878

119901minus119868119901)2 keeps the

main image structure similarly The corresponding solver of(5) will be given in the following

Solving For the optimization problem (5) it is a nonde-terministic polynomial-time (NP) hard problem to find theglobal optimal solution Motivated by Wang et al [32]a splitting scheme is proposed to solve the optimizationproblem Two auxiliary variables ℎ

119901and V119901are introduced to

substitute 120597119909119878119901and 120597119910119878119901 Equation (5) can be rewritten as

min119878ℎVsum

119901

(119878119901minus 119868119901)2

+ 120582 sdot 119862 (ℎ V)

+ 120573 ((120597119909119878119901minus ℎ)2

+ (120597119910119878119901minus V)2

)

(6)


(a) (b) (c)






119901 119907(119894)

119901via (10) when fixing 119878(119894)


119901

(5) Update 120573 larr 120581120573(6) 119894 larr 119894 + 1




119909119878119901 120597119910119878119901) For solving



min119878

sum

119901

(119878119901minus 119868119901)2

+ 120573 ((120597119909119878119901minus ℎ)2

+ (120597119910119878119901minus V)2

) (7)


119878 = Fminus1

times (

I + 120573 (F (120597119909)lowastF (ℎ) +F (120597119910)

lowast

F (V))

F (1) + 120573 (F (120597119909)lowastF (120597119909) +F (120597

119910)lowast

F (120597119910))

)

(8)



minℎVsum

119901

120573 ((120597119909119878119901minus ℎ)2

+ (120597119910119878119901minus V)2

) + 120582 sdot 119862 (ℎ V) (9)

where 119862(ℎ V) = sum119901119867(|ℎ119901| + |V119901|)119867(|ℎ

119901| + |V119901|) = 1 if |ℎ

119901| +

|V119901| = 0 and 119867(|ℎ


rewritten as

sum

119901

minℎ119901 V119901120573 ((120597119909119878119901minus ℎ)2

+ (120597119910119878119901minus V)2

) + 120582 sdot 119867 (10038161003816100381610038161003816ℎ119901

10038161003816100381610038161003816+10038161003816100381610038161003816V119901

10038161003816100381610038161003816)

(10)



119901| + |V119901|) reaches its


(ℎ119901 V119901) =

(0 0) (120597119909119878119901)2

+ (120597119910119878119901)2

le120582

120573

(120597119909119878119901 120597119910119878119901) otherwise

(11)












0smoothing





0rdquo is the




0rdquo is


0




0image


3 Results


5 and 1205730= 2120582




(c) 01rsquoTIP

(c) 01rsquoTIP

(d)06rsquoTIP

(d)06rsquoTIP

(e) 07rsquoTIP

(e) 07rsquoTIP

(a) LR

(a) LR

(b) Bicubic

(b) Bicubic

(f) Ours

(f) Ours






4 Conclusions



0smoothing method




Acknowledgment


References










































Volume 2014




Journal of











Journal of


Function Spaces






Algebra










(a) (b) (c)






119901 119907(119894)

119901via (10) when fixing 119878(119894)


119901

(5) Update 120573 larr 120581120573(6) 119894 larr 119894 + 1




119909119878119901 120597119910119878119901) For solving



min119878

sum

119901

(119878119901minus 119868119901)2

+ 120573 ((120597119909119878119901minus ℎ)2

+ (120597119910119878119901minus V)2

) (7)


119878 = Fminus1

times (

I + 120573 (F (120597119909)lowastF (ℎ) +F (120597119910)

lowast

F (V))

F (1) + 120573 (F (120597119909)lowastF (120597119909) +F (120597

119910)lowast

F (120597119910))

)

(8)



minℎVsum

119901

120573 ((120597119909119878119901minus ℎ)2

+ (120597119910119878119901minus V)2

) + 120582 sdot 119862 (ℎ V) (9)

where 119862(ℎ V) = sum119901119867(|ℎ119901| + |V119901|)119867(|ℎ

119901| + |V119901|) = 1 if |ℎ

119901| +

|V119901| = 0 and 119867(|ℎ


rewritten as

sum

119901

minℎ119901 V119901120573 ((120597119909119878119901minus ℎ)2

+ (120597119910119878119901minus V)2

) + 120582 sdot 119867 (10038161003816100381610038161003816ℎ119901

10038161003816100381610038161003816+10038161003816100381610038161003816V119901

10038161003816100381610038161003816)

(10)



119901| + |V119901|) reaches its


(ℎ119901 V119901) =

(0 0) (120597119909119878119901)2

+ (120597119910119878119901)2

le120582

120573

(120597119909119878119901 120597119910119878119901) otherwise

(11)












0smoothing





0rdquo is the




0rdquo is


0




0image


3 Results


5 and 1205730= 2120582




(c) 01rsquoTIP

(c) 01rsquoTIP

(d)06rsquoTIP

(d)06rsquoTIP

(e) 07rsquoTIP

(e) 07rsquoTIP

(a) LR

(a) LR

(b) Bicubic

(b) Bicubic

(f) Ours

(f) Ours






4 Conclusions



0smoothing method




Acknowledgment


References










































Volume 2014




Journal of











Journal of


Function Spaces






Algebra


















0smoothing





0rdquo is the




0rdquo is


0




0image


3 Results


5 and 1205730= 2120582




(c) 01rsquoTIP

(c) 01rsquoTIP

(d)06rsquoTIP

(d)06rsquoTIP

(e) 07rsquoTIP

(e) 07rsquoTIP

(a) LR

(a) LR

(b) Bicubic

(b) Bicubic

(f) Ours

(f) Ours






4 Conclusions



0smoothing method




Acknowledgment


References










































Volume 2014




Journal of











Journal of


Function Spaces






Algebra










(c) 01rsquoTIP

(c) 01rsquoTIP

(d)06rsquoTIP

(d)06rsquoTIP

(e) 07rsquoTIP

(e) 07rsquoTIP

(a) LR

(a) LR

(b) Bicubic

(b) Bicubic

(f) Ours

(f) Ours






4 Conclusions



0smoothing method




Acknowledgment


References










































Volume 2014




Journal of











Journal of


Function Spaces






Algebra













4 Conclusions



0smoothing method




Acknowledgment


References










































Volume 2014




Journal of











Journal of


Function Spaces






Algebra

































Volume 2014




Journal of











Journal of


Function Spaces






Algebra









research article single image super-resolution via image...

Documents