motion adaptive deinterlacing combining with texture detection.pdf

7/27/2019 Motion Adaptive Deinterlacing Combining with Texture Detection.pdf

1/4

IEEE Int. Workshop VLSI Design Video Tech.Suzhou, China, May 28-30,2005

Mo tion Adaptive D einterlacing Com bining with Texture Detectionand Its FPGA Implem entationYung Yuhong, Chen Yingqi, hung Wenjun

Shanghai JiaoTong U niversityHaoran Ed. 15F Huashan Rd. No. 1954Shanghai 200030 P.R.China

ABSTRACTA same-parity 4-field based motion adaptive deinterlacingalgorithm combining with texture detection is presented inthis paper. Comparing with the existing low-enddeinterlacing algorithm, this method greatly improvesimage quality for both static and fast moving objects due tohigh motion detection accuracy. Its hardware architectureis also simpler than high-end deinterlacer architecture, e.g.motion estimation compensation deinterlacer. Its FPGAimplementation is also presented.

1. INTRODUCTIONInterlace scanning in high resolution display devices maycause some visual artifacts. There are some high-enddeinterlacing methods which use motion-estimation andadaptive interpolation algorithm and some are object-basedtrue-motion estimation algorithm[2]-[6]. All of thesemethods can provide better deinterlacing quality. However,considering the hardware cost, there are alsomany low-enddeinterlacing methods for consumer electronics. BOB andWeave[]] are two low complexity deinterlacing methods.BOB is an intra-interpolation method to reconstruct oneprogressive frame that the vertical resolution is halved andthe image is blurred. Weave deinterlacing methodcombines directly two interlaced fields into oneprogressive frame. The line-crawling effects occur in themotion area. Some motion adaptive deinterlacingtechniques have been presented to improve imagequality[h][7]. We also presented an motion adaptivedeinterlacing algorithm with 2 hierarchical weightedaveragc before[X]. Based on our previous work and somereferences, we present a high quality deinterlacingalgorithm o f the same-parity 4-field based motion adaptivedeinterlacer with texture detection. Its hardwarearchitecture and available memory access arbitrationmethod with fixed order will also be described.The overview of the motion adaptive deinterlacingmethod and software simulation result is shown in section2 and section 3 . Section 4 presents its hardwarearchitecture and FPGA implementation result. Section 5

0-7803-9005-9/05/ 20.000 2 0 0 5 IEEE 316

gives the conclusion.2. THE PROPOSED MOTION ADAPTIVEDEINTERLACING METHOD

Our previous motion adaptive deinterlacing method[8] uses2 hierarchical weighted average. That is, the intra-fieldinterpolation of 2-D Cubic interpolation and edgeinterpolation according to edge detection is calculated first,then the weighted average result of intra-field interpolationand inter-field interpolation is caiculated according tomotion detection coefficient. By using 2-field motiondetection method, we have to average the upper-line andthe lower-line in the previous field comparing with thecurrent line in the current field due to the different parity ofthe 2 fields. The motion detection error sometimes is muchhigh. So we consider to use same parity fields motiondetection.The 4-field horizontal motion detection method from5 directions is shown in fig. 1[7]. Block matching is donebetween the forward field and the backward field by 1x3block size from five directions. If the m inimum differenceof block matching is smaller than the threshold, and thepixel difference between the forward-forward field and thecurrent field is also smaller than the threshold, the temporalinterpolation will be adopted.

The threshold value used to detect the motion area iscalculated according to [7] which is based on the principlethat human eyes are less sensitive to lighter or darker area


2/4

than gray area, so the threshold at those pixels should bemuch smaller than the threshold at gray color pixels. Wedetermine the motion area when the field difference islarger than the threshold value. When the field difference issmaller than the threshold value, the static area is found.Equation 1 presents the simple threshold adjustingmechanism for motion detection. For an 8bit gray scalepicture, 255 and 0 mean the white and black color,respectively. Experimental result shows that when thecurrent pixel value is 255 or 0, which is not sensitive forhuman eyes, the threshold value is 20. When the currentpixel value is 127, which has high sensitivity, the thresholdvalue is 10. [7]

I O .I / / \ [ I / > / L / = O l 2 X \ \ h I l C I .: 12s

A s the luminance of the same parity fields in the samestatic area may have differences that may not bedistinguished by eyes, that is, their texture is inconsistent.The threshold value given should be high enough to detectthe static area. But if we raise the threshold value,sometimes the true motion area cannot be detectedcorrectly, especially in fast moving areas. Sometimes thethreshold trade-off cannot be given for acquiring goodquality results of both static and fast moving sequences.In order to solve the problem, we propose to utilizetexture consistency detection. Because the texture must beinconsistent in fast moving areas of forward and backwardfields in case their temporaI differences in same parityfields are under the original threshold, these areas cannotbe detected as static areas. So by detecting textureconsistency of forward and backward fields, motiondetection accuracy can be improved.Only when temporal field difference is smaller thanthe threshold value and the texture of the 2 fields in thatdirection is consistent, the static area is found. Otherwisewe use motion adaptive weighted interpolation of thecurrent field and backward field.

The block size of texture detection is 1x9. Fourtexture grads(n= - 1 0, 1 2) of every other pixel of eachblock in forward and backward fields are calculatedrespectively. The texture consistency of the two fields arematched along the five directions which is the sam e as thatin motion detection. The four texture grads calculation ineach field is shown in fig. 2and equation 2, 3:

Figure 2 . Backwardforward field texture detection

- F x w i d t h y +2 n+1) n = 1,0,1,2 2 )(3)

Gvad n,dir,x,y),,wo,dJi.rd = F x * widrh + y +dir + 2 * n )~ F x w id t h + + dir I * n Iwhere F( -) is luminance of forward or backward field.

(x,y) is the center pixel of the current block. width is thepixel number of a line of each field. dir is the direction oftexture consistency detection from -2 to 2. That is, wedefine the no. of pixels in 1x9 block from 1 to 9. The fourtexture grads in each field are pixel value difference of 4pairs: 1-3, 3-5, 5-7, 7-9. If all the absolute value of texturegrads of the forward and backward fields in the samedirection of motion detection are lower than a fixedthreshold or the corresponding grads have the same sign ofthe two fields, then we a ssume the texture of the two fieldsin this direction is consistent. Otherwise it is inconsistent.The block diagram of the deinterlacing method is

shown in fig. 3. Three-field buffers are used to store thereference data from 3 previous fields. The ELA moduledoes the directional edge interpolation according to thecurrent-field information[7]. The same-parity 4-fieldhorizontal motion detection and backward\fonvard fieldstexture consistency detection calculate the differencesbetween the forward-forward field and the current field,and the difference between the forward field and thebackward field. The field difference will be sent to thethreshold-adjusting module. According to the pixel valueof the current field, the threshold-adjusting module willprovide an adaptive threshold value to produce the motioninformation. The decision block receives the motioninformation and texture consistency information ofbackward and forward fields, and then selects the forwardfield in the static area or motion adaptive weightedinterpolation of ELA of current field and backward fieldaccording to motion adaptive coefficient which isdescribed behind.So if static area is detected, the average of forwardfield and backward field in the detected direction isselected. Otherwise if the temporal field difference value ishigher than threshold or the texture of backward andforward fields is inconsistent, the static area cannot befound. Then we use motion adaptive weightedinterpolation of the current field and backward field.Motion adaptive coefficient is selected from theprecalculated LUT which is derived from experimentalstatistics according to the difference of the average ofupper and lower pixel of the interpolated pixel in currentfield and the corresponding pixel in the backward field.The difference is the input of the coefficient LUT that canoutput motion adaptive coefficient. The resultedinterpolation value is as follows:

n = -l,0,1,2

Mapresult=( 1 Ma-coef)* E dge-current+M a c o e P -back wa rd 4)

317


3/4

M a c o e f is motion adaptive coefficient kom LUT.Edge-current is directional edge interpolation in currentfield . fbackward i s the corresponding pixel value inbackward fieId.Finally, the current field and the interpolation field aremerged into the progressive frame.

sckrard l i c l d .

Figure 3. Block diagram o f proposed deinterlacingmethod3. ALGORITHM SIMULATION RESULTS

The sequences of racing car and pendulum is simulatedusing the motion adaptive deinterlacing method presentedin [7] and our proposed method respectively. And thesequence of pendulum is also simulated using our previousmethod[S]. In pendulum sequence, the letters of 0 andK are only presented seperately in different fields. Theracing car sequence has a fast moving background.With the method of [7], the fast motion sequence ofracing car cannot be detected correctly. There appearsmany noises in fast moving background(see fig. 4). Itseems that threshold is too high. Because the difference ofthe two same parity fields is below the threshold, thesequence must be detected to be static. Then the motionadaptive deinterlacing is inter-field interpolation withwrong neighboring field value. But if we reduce thethreshold value, the static letter of 'O'and 'K' in pendulumsequence cannot be reserved in all resulted frame (see fig.5 ) because the difference of the two fields in samedisparity in static area is sometimes larger than thethreshold. Then the static area is detected to be in motionstate and deinterlacing is done by intra-field interpolation.So flicker effect will exist in letters area. The resultedframe of pendulum using our previous method has thesame effect.

Using our proposed methods these problems can besolved in a way. Good quality can be acquired in bothstatic own in fig. 6(a) (6).

Figure 4. Deinterlacing of fast motion sequence:racingcar using method of [7]

Figure 5 Deinterlacing of pendulum with static letter O&Rusing method of [7] our previous method (consecutivetwo frames)

(ai racing car

(b) pendulum .Figure 6 2 deinterlacing sequences by our proposed318


4/4

method4. ARCHITECTURE DESIGNAND FPGAIMPLEMENTATION

The architecture of the deinterlacer is shown in fig, 7There are six main units: input unit, 4-field motiondetection and texture consistcncy detection unit,SDRAMcontroller unit, FIFO unit, ELA intra-fielddeinterlacing unit and ouput buffer unit,i - F i e l d Uniion ,

Figure 7. Architecture of 4-field based deinterlacerwhere k is the motion adaptive weighted coefficientof current field and backward field.Input unit has the same function as we describedbefore[8]. It accepts 2 channels or channel 4:2:2 videoinput. Meanwhile the input control module controls theglobal frameifield synchronization, It extracts theTRS(Timing Reference Signal) information from inputvideo stream that corresponds to SM PTE or BT.656 or getthe synchronous information f iom output F-in,Vin,H-insignals io synchronize the whole IC.

ELA Intra-field deinterlacing unit perfo rms edgedetection and intra-field ELA interpolation in the currentfield. 4-field motion detection and texture consistencydetection unit performs the key function that receives oneline from each of the 4 fields during the same period. Thecalculation is described above. W hen static area is detected,it outputs the average of corresponding line blocks inbackward field and forward field. Otherwise, it outputs thebackward field data line by line. Meanwhile the motionadaptive coefficients k and 1-k can also be a cquired whichhas been decribed above. If the static area is found, k is setto zero. Otherwise k is fiom zero to one. Threefields--current field, forward field and forward forwardfield are stored in SDRAM. So the four lines of the 4 fieldsshould be fetched f i o m S D M M and input streamsimultaneous1 .We adopt the time sharing memory access scheme.SD RA MC ontro ller module arbitrates the memory accessrequests and generates memory access signals. It uses fixedorder of the four FIFO modules access requests. Foravoiding overflowiunderflow of each FIFO, evaluation

considering overhead of SDRAM access and simulationshows that each FIFO depth should be 64 and themaximum of SDRAM interface clock is lOOMHz forHDTV video format.We use Xilinx virtexII 4000 to implement thedeinterlacer because we will combine other functionmodules such as scaler and video enhancement together.External SDRAM is 32bit 64Mbit. The resource usage ofFPGA is shown in table I. o the hardware implementationis simple an d cost efficient.Table I . FPGA Resource UsageResource Usage Rate

External IOBs 17%Slices 39RAMB 16s I 105. CONCLUSION

We propose a new motion adaptive deinterlacing methodcombining with texture consistency detection to improveimage quality. It is based on same-parity 4-field horizontalmotion detection. Experimental results of the comparisonsbetween the proposed method and our previous methodand the method in [7] are given above. It shows that the.method we propose in this paper is better for both staticand fast moving pictures. Its hardware architecture andFPGA implementation result are also presented with lowcomplexity and high-speed processing capability. We use asimple and efficient memory arbitration algorithm. It isvery cost-efficient.

6. REFERENCES[ I ] http://nickygukdes.digital digest.com/index.htm121 Kenju Sugiyamaand Hiroya Nakamura, A Method ofDeinterlacing with Motion Compensated Interpolation, IEEETrand Consumer Elec.,vol.45, no. 3, pp61 1-6 16, August 1999.[3] Dang-Jiang Wang and Jin-Jang Leou, A New Approach toVideo Format Conversion Using Bidirectional Motion Estimationand Hybrid Error Concealment, Joumal of information Scienceand Engineering 17, pp753-777,2001,[4] Jed Deame, Motion Compensated Deinterlacing: The Keyto the Digital Video Transition, SMPTE 141st TechnicalConference in NY, November 19-22, 1999.[5] D.Van de Ville, WPhilips, nd l.Lemahieu, MotionCompensated De-interlacing for Both Real T i m e Video and St i l lImages, International Conference on Image Processing, Vol. 2,[6] G . De Haan, E.B.Bellers, Deinterlacing-an Overview,Proceedings of the IEEE, vo1.86, Issue:9, pp 1839-1857, Sept.1998.[7]Motion Adaptive Interpolation with Horizontal MotionDetection for Deinterlacing, IEEE Transactions on ConsumerElectronics, Voi, 49 No.4 November 2003[S Ya ng Y uhong, Zhang Wenjun, Adaptive DeinterlacingAlgorithm and Its Circuit Design, Journal of DataAcquisition&Processing,Vol.19 No.3,pp334-338,2004

pp 680-683,2000,

Shyh-Feng Lin, Yu-Ling Chang, and Liang-Gee Chen,

3 19
http://nickygukdes.digital-digest.com/index.htmhttp://nickygukdes.digital-digest.com/index.htm

motion adaptive deinterlacing combining with texture detection.pdf

Documents