Revisiting the JPEG-LS prediction scheme

J.Jiang, B.Guo and S.Y.Yang

Abstract: The authors investigate the prediction scheme of JPEG-LS, the latest JPEG standard for lossless/near lossless image compression. They show that it is not sufficient to consider only horizontal and vertical edges in constructing predictive values. As a result, they propose an additional diagonal edge detection scheme to achieve better prediction accuracy and hence provide potential for further improvement. Experiments show that, in terms of mean-square-error values, the proposed scheme outperforms the existing JPEG-LS prediction for all images tested, while the complexity of the overall algorithm is maintained at a similar level.

1 Introduction

JPEG-LS [l] is the latest international standard for lossless and near lossless still image compression developed by JPEG, the joint expert group for image coding and compres- sion. Similar to previous JPEG standards [2], JPEG-LS was developed by a combination of optimised coding schemes selected from a number of successful candidate algorithms on a world-wide competition basis. The draft document was distributed in 1997 [ 11 and the formal standard issued in 1998 [3]. The main compression techniques proposed in JPEG-LS can be summarised as: (i) run-length coding; (ii) nonlinear prediction; (iii) context-based statistics model- ling; and (iv) Golomb entropy coding [4].

For an input image, a prediction scheme is first operated to decide whether the run-length compression mode or the predictive coding mode should be selected to encode the current pixel, depending on the values of those previously encoded pixels in a surrounding neighbourhood. The neighbourhood is represented by a predictive template of four pixels as illustrated in Fig. 1. To reduce the computa- tional cost for statistical modelling and for selection of an appropriate coding mode. JPEG-LS proposed the follow- ing three A values to implement the local texture analysis.

A , = d - b ; A 2 z b - c ; A ~ = c - u (1)

As a result, the consideration of the four pixel intensity values can be reduced to the consideration of three A values.

JPEG-LS also includes a near lossless compression mode to address applications, where higher compression efficiency is required and small loss of information can be tolerated. With this compression mode, information loss is introduced in JPEG-LS as a fixed, pre-determined constant value represented by a parameter NEAR [l, 5, 61. If all four neighbouring pixel values are the same (lossless) or their differences ( A values given in eqn. 1) are less than NEAR (near lossless), it is indicated that the local region

0 IEE, 2000 IEE Proceedings online no. 20000767 DOJ: 10. 1049/ip-vis:20000767 Paper received 10th April 2000 J. Jiang is with the School of Computing, University of Glamorgan, Pontypridd CF37 IDL, UK B. Guo and S.Y. Yang are with the Signal Processing Laboratory, Institute of Acoustics, Chinese Academy of Sciences, People’s Republic of China

IEE Prm-Vis. Image Signal Process., El. 147, No. 6, December ZOO0

surrounding the pixel to be encoded is of smooth texture. Hence, run-length coding is selected to encode the next sequence of pixels until the run is broken. Otherwise, the texture may not be that smooth and thus a prediction-based entropy coding is selected. This type of near lossless encoding has the effect, at the decoding end, that the reconstructed pixels may have a maximum distortion of NEAR in comparison with their original values. Hence, the selection of its coding mode in JPEG-LS can be summarised below

codingmode = run-length if A , , A2 and A, < NEAR

predictive coding otherwise

Prediction is designed by exploiting a simple local texture analysis among the three context pixels, a, b and c, inside the predictive template given in Fig. 1. Specifically, comparisons of intensity values of the three surrounding pixels are made to see if any horizontal edge or vertical edge can be detected. When an edge is detected among the three pixels, the pixel that is not on the edge geographically will be taken as the predictive value. Otherwise, the predictive value will be an offset value drawn from all three pixels. The detailed design of the scheme can be summarised by the following pseudocodes according to the predictive pattern shown in Fig. 1:

if (c >= max(a, b)) P = min(a, b); else{

if (c i= min(a, b)) P = max(a, b); else P = a + b - c;

where max(a, b) and min(a, b) stand for the maximum and minimum values among pixels a and b, respectively.

Fig. 1 JPEG-LS predictive template

2 Proposed prediction scheme

To assess the existing prediction scheme in JPEG-LS, further analysis of the above pseudocodes can be made to reveal the following facts:

e An edge would be detected among the three pixels when either c > =max(a, b) or c < =min(a, b) is satisfied. Whether the edge is horizontal or vertical depends on which pixel is the maximum or minimum value between a and b. When c > = max(a, 6 ) is satisfied, as an example, max(a, b) = a give us a vertical edge and max(a, b) = b a horizontal edge. Similarly, the condition, c < = min(a, b), would give us a horizontal edge if min(a, b)=b and a vertical edge if min(a, b) =a. Fig. 2 shows two examples to illustrate such an edge detection process. From the geographical arrangement shown in Fig. 1, it can be seen that a logical selection of the predictive value would be the pixel, which is not on the detected edge.

All other cases are represented by the condition: min(a, b) < c < max(a, 6). Under this circumstance, it is difficult to construct a predictive value to cover all possi- bilities in the local texture. This is because there is no clear justification to determine whether or not there is an edge and what the edge should be even if it exists. This is because any positive analysis has to be dependent on the unknown pixel value x. For instance, a horizontal edge can be detected only when x is found to be closer to a, or a vertical edge can be detected if x is closer to b. In fact, the choice of a+ b - c is an excellent balance among all possible situations. Assuming a > b, as an example, this would give us: b < c < a. Under this circumstance, three possible cases of predicting x can be identified and analysed. Details are given below:

(i) A vertical edge exists in the predictive template. In this case, it can be seen from Fig. 1 that c must be close to a to have a vertical edge and hence the value of x is very likely to be close to b due to their geographical positions. Since a - c is small, the prediction would be close when a - c is added to b. (ii) A horizontal edge exists in the predictive template. To have a horizontal edge, c must be close to b and x is likely to be close to a. In this case, a - c must be very large and, when added to b, the difference between b and x would be compensated to a large extent. Therefore, the prediction would still be good. (iii) No edge exists inside the predictive template. In this case, it can be easily seen that a - c+b would make a very good prediction since the local texture is relatively smooth.

Similar analysis can also be made when the condition a c: b is satisfied. Therefore, the prediction a +b - c can be

a b

Fig. 2 Local edge detection a Vertical edge is detected b Horizontal edge is detected

576

viewed as a self-adaptive process to minimise the predic- tive error for all possible cases.

0 from the above analysis, it can be verified that the prediction scheme adopted in JPEG-LS only considers the detection of a horizontal edge or a vertical edge among the predictive template to produce a predictive value. This is logical in the sense that the variety of intensity values around a neighbourhood area can always be described by the two types of edges, when the viewing angle is limited to the local predictive template. In other words, combinations of many vertical and horizontal edges can produce a line with arbitrary shape or edges in any image. However, what matters here is whether the descrip- tion by the two edges will be accurate enough to minimise the predictive errors. In cases where a diagonal edge exists inside the predictive template, it can be expected that the errors produced may be unnecessarily high.

To further improve the prediction scheme, therefore, one of the possibilities is to consider the diagonal edge detection among the three pixels inside the predictive template. As the diagonal distance between a and b is larger than any other distance inside the template, and the diagonal edge detected can also be interpreted as either a vertical or a horizontal edge in most cases, it can be anticipated that the occurrence of a true diagonal edge would be much less frequent compared with that of vertical or horizontal edges. Consequently, the possible improvement would be depen- dent on the number of cases where a true diagonal edge exists inside the local areas of the input image. Essentially, the possibility of improvement can be explored in two directions: (i) to accurately detect the true diagonal edges; and (ii) to correctly construct the predictive value, based on the diagonal edge detected.

In addition, to detect a diagonal edge, only two possi- bilities (45" and 135") exist among the three pixels, a, b and e. It can also be assumed that there is no edge that is one pixel wide inside the predictive pattern, or if any, such an edge can be ignored by our scheme since it is hardly possible to produce any meaningful prediction from this type of edge. Hence, all the possibilities for the existence of a diagonal edge can be summarised in Fig. 3, where high and low represent large and small intensity values, respec- tively.

Considering all the possibilities in the light of the existing prediction scheme designed in JPEG-LS, a detailed analysis of the diagonal edge detection and pre- diction can be carried out for each of the predictive con- texts given in the pseudocodes, i.e., (i) min(a, b ) i c s max(a, b); (ii) c 3 max(a, b); and (iii) c 5 min(a, b).

First, when the condition min(a, 6 ) 5 c 5 max(a, 6 ) is satisfied, it can be seen that only a 135" diagonal edge can possibly exist, which naturally leads to the two possibilities

a b

C d

Fig. 3 Possible forms of a diagonal edge

IEE Proc-Vis. Image Signal Process., El. 147, No. 6, December 2000

shown in Fig. 3c and Fig. 3d, depending on which of a o r b is larger. Under this circumstance, as we mentioned before, the specific edge type can only be determined by the unknown pixel x. For the case shown in Fig. 3c, the fact that the unknown pixel x is close to a, b or c would give us a horizontal edge, a vertical edge or a diagonal edge, respectively. This can be illustrated in Fig. 4a. For the case of Fig. 3d, similar analysis can be carried out as shown in Fig. 4b. Therefore, it is extremely difficult to detect the edge with reasonable accuracy since the correct detection has to be dependent on the value of the unknown pixel n. Hence, it is unlikely to have any further improve- ment with this predictive context without jeopardising the good predictive results already achieved by JPEG-LS.

Secondly, with the condition c>max(a, b), it is not difficult to see that this condition would only lead to Fig. 3a if there exists one diagonal edge. Depending on the specific values of a and b, the existing JPEG-LS scheme would give us either a horizontal edge or a vertical edge among the three pixels. However, if there exists a diagonal edge, it could be included in both cases. This is shown in Fig. 5. The principle is that we need to detect the diagonal edge with the best possible accuracy from the information given by the available pixels at both encoding and decod- ing ends, not by the unknown pixel x.

Assuming that a diagonal edge exists, it would be helphl to examine what conditions should be satisfied to have such an edge. First, it should be established that either a vertical edge or a horizontal edge cannot coexist with the diagonal edge, or any coexistence should be ignored. This is because, in this circumstance, it is impossible to produce an accurate prediction from one type of edge against the other since the situation contains too much uncertainty. To this end, we conclude that, there should be a certain gradient between the intensity values of c and max(a, 6) to allow a diagonal edge be established and the possibility of a verticalhorizontal edge be minimised. This gives us the following condition:

c - max(a, 6) > threshold, (3)

For convenience of description, c, a, b, etc. are directly used to represent their intensity values in the rest of this paper.

I 1 xctose toa

a

x close to c

a

C

x close to a

b /

\N x close to c

b

Fig. 4

LEE Proc.-Vis. Image Signal Process., Vol. 147, No. 6, December 2000

Analysis for the context min(a, b) 5 c 5 max(a, b)

C b

/ \ x<min(a,b)

a> b Fig. 5 Edge detection for c 2 max(a, b)

Secondly, from the nature of the diagonal edge, other conditions can be easily identified as given below:

x < min(a, b) and d < b (4) The first part of expr. 4 does not produce any usehl information since it involves the unknown pixel x. As a result, only the second part can be used to detect the diagonal edge. In the rest of this paper, when expr. 4 is referred to, it only means the second part, i.e. d < b.

On the other hand when such a diagonal edge is detected, the predictive error produced by the existing JPEG-LS scheme is large. This is because, as shown in Fig. 3a, the predictive value produced by JPEG-LS in this case is always min(a, b), yet the existence of a diagonal edge leads to the fact that there is a large difference between x and min(a, b). Therefore, further improvement over JPEG-LS is possible, if the detection and prediction can be designed accordingly.

When a c b is satisfied, however, it is difficult to construct a predictive value even when a diagonal edge is detected accurately by exprs. 3 and 4. Although x < a is expected, there is no correlated pixel on this row to compensate for the difference: x - a. Hence, we abandon the attempt to propose any new prediction for the time being, which can be regarded as part of the further research needed in this direction. This would leave us with the case: a > b.

Since the potential for improvement over JPEG-LS lies in the fact that there could be a large difference between x and b (min(a, b)), it would be essential for the success of our proposed scheme to detect such a difference. Accord- ing to the pattern given in Fig. 3a, the difference between b and x can be indirectly detected by the difference between a and x, assuming exprs. 3 and 4 are satisfied. Hence, if we introduce a gradient detection between a and b, a large difference between a and n would become likely, consider- ing other gradients we have already detected. As a result, such a detection can be constructed as follows:

a - b 2 threshold2 ( 5 )

When all the above conditions are satisfied, it can be established that the existence of a diagonal edge, as shown in Fig. 3a, would produce a large difference between x and b. Thus, the predictive value b produced by PEG-LS becomes distant from x and the predictive error is large. To rectify the situation, a certain level of compensation can be introduced by considering the value of pixel d, since the detection of a true diagonal edge indicates a trend of d being smaller than b and c. This can also be illustrated in Fig. 3a. As a result, the new predictive value under this circumstance is

d f b p = - 2

Eqn. 6 brings the predictive value closer to the unknown pixel x. Table 1 illustrate a sample of pixel intensity values and their predictive errors produced by JPEG-LS for the image ‘Lena’. In both cases, it can be seen that, when a

571

Table 1: Sample of pixel intensity values in ’Lena‘ and corresponding predictive errors

Pixel value Predictive error

197 199 196 198 196 143 52

195 195 194 198 185 96 50

191 193 196 198 154 69 50

187 193 198 194 114 54 54

183 192 199 174 70 55 59

185 195 . 197 141 56 49 49

191 201 187 92 47 48 50

196 203 174 68 48 52 48

199 200 148 55 46 46 49

204 193 110 49 49 48 52

205 171 69 49 48 55 49

200 142 56 50 51 55 58

188 98 53 53 50 48 52

165 64 52 50 48 53 50

NA

NA

NA

NA

NA

NA

NA

NA

NA

NA

NA

NA

NA

NA

NA

- 2

2

4

3

3

7

2

- 3

- 11

- 23

- 29

- 44

- 34

NA

- 1

3

2

2

- 2 - 14

- 15

- 26

- 38

- 41

- 13

- 3

- 1

NA

2

0 - 4

- 21

- 33

- 49

- 24

- 13

- 6

0 1

3

- 2

NA

- 11

- 31

- 40

- 44

- 14

- 9

1

- 2

3

- 1

2

- 3

- 2

NA

- 47

- 27

- 15

1

- 6

1

4

- 4

- 1

7

0

- 6

5

NA

NA

NA

NA

NA

NA

NA

NA

NA

NA

NA

NA

NA

NA

true diagonal edge does exist, the existing prediction scheme fails to produce minimum predictive errors.

Finally, corresponding to the condition: c 5 min(a, b), we would have the case given in Fig. 3b. Similar analysis can be made to reveal that if a true diagonal edge exists among the three pixels, a, b and c, the predictive value, max(a, b) produced by JPEG-LS, may not be large enough since a significant difference between x and max(a, b) could be expected. Hence additional improvement can also be achieved, if the diagonal edge is detected correctly.

To detect a diagonal edge, first, a large difference or a gradient between c and min(a, 6 ) should exist, reducing the possibility for the coexistence of any other type of edges (horizontal or vertical). This gives us the following condi- tion:

min(a, 6 ) - c 2 threshold, (7)

Secondly, to exclude any other type of edges, the differ- ence between a and b should be constrained as given below:

I b - a I I. threshold, (8)

Thirdly, we expect that the value of d should be greater than b, to ensure that there is a large difference between x and the predictive value max(a, 6). However, it is desirable to constrain the difference, d - 6 , to a certain limit. This is because, if the difference is too large, the edge may be less than 45” and hence x could be included in the edge, as illustrated in Fig. 6. As a result, the following condition can be added to detect the diagonal edge:

threshold5 5 d - b 5 threshold6 (9)

When all the conditions specified in eqns. 7-9 are satisfied, it is expected that the predictive value max(a, b) may not

Table 2: Threshold values

Thresholds 1 2 3 4 5 6

Empirical values 10 5 5 10 10 50

be large enough to be close to x and thus the following predictive value is proposed:

L.

All the threshold values are to be determined empirically. From our testing, we propose the values given in Table 2. In summary, the proposed prediction scheme can be shown by the How-chart given in Fig. 7.

3 Experiments

To test the proposed algorithm, we implemented the prediction scheme as shown in Fig. 7 and ran the program on four test images: ‘Lena’, ‘Boat’, ‘Camera’, and ‘Clown’. To ensure a fair comparison with the existing JPEG-LS scheme, we assessed the performance of the proposed scheme in two measurements: (i) the MSE (mean-square-error) between each pixel to be encoded and its predictive value; and (ii) the compression ratio. The first measurement is designed to assess the accuracy of the proposed prediction scheme benchmarked by the exist- ing state-of-the-art prediction adopted in JPEG-LS, the

Fig. 6

578

Example ofd on the diagonal edge Fig. 7 Flow chart for proposed prediction scheme

IEE Proc.-viS. Image SignaI Process., h l . 147, No. 6, December 2000

Table 3: Experimental results

Samples Size JPEG-LS Proposed JPEG-LS Proposed MSE values MSE values compression ratio compression ratio

Lena 512 x 512 37.435 35.104 1.991

Boat 512 x 512 58.174 56.964 1.881

Camera 256 x 256 215.414 214.457 1.849

Clown 256 x 256 93.7 15 92.008 1.702

1.991

1.881

1.849

1.702

latest JPEG standard for losslessinear lossless image compression. The second measurement is designed to see if the proposed scheme has any impact on the compression performance without any change to the entire coding scheme, including statistics modelling and Golomb entropy coding. To ensure a fair comparison, both algo- rithms, the proposed and JPEG-LS, have exactly the same operations including coding-mode selection, statistics modelling and entropy coding, apart from the prediction being different. With these settings, the experimental results for all image samples can be summarised as shown in Table 3.

From this table, it can be seen that, for all image samples tested, the proposed scheme outperforms JPEG-LS in terms of MSE values. As analysed earlier, the extent of the improvement depends on the image content, i.e. the number of true diagonal edges, which may exist inside the input image.

Another interesting aspect, which can be observed from Table 3, is that, although the prediction scheme is improved in terms of accuracy measured by MSE values, the compression ratios stay the same. This is expected due to the fact that the entropy coding length is determined by statistical modelling [7], yet this part of the operation is not revised in the light of the proposed diagonal edge detec- tion. In this sense, a smaller predictive error may not necessarily produce higher compression efficiency. The decisive factor for compression efficiency is the accuracy with which the statistical modelling produces statistical information to drive the entropy coding [SI. However, when predictive error is minimised, the direct and positive effect upon statistical modelling would be significant, since the statistical distribution of those errors would become more focused around its mean value as opposed to being scattered [7]. With JPEG-LS, the specific advantage can be illustrated that the number of context quantisation regions could be reduced and more probabilities could be assigned around the centre of the statistical distribution. Another impact on compression efficiency by smaller predictive errors can be seen with the near lossless compression mode in JPEG-LS. The near lossless mode requires small quan- tisation of those predictive errors to produce higher compression ratios. To this end, the reconstructed image quality could benefit by introducing a smaller quantisation step corresponding to the smaller predictive errors [5, 61.

To further illustrate the improvement over JPEG-LS of the proposed scheme, we also present those predictive error values as an image and thus enable a visual compar- ison between, JPEG-LS and the proposed scheme. The advantage of so doing is to see exactly where the improve- ment is, since the MSE value only indicates an overall average and often fails to reveal any useful information about the error distribution. Additional gain can also be obtained to certify that while improvement is evidenced by MSE values, no loss is made in those areas where only vertical and horizontal edges exist by our proposed

IEE Proc.-Vis. linage Signal Process., Vol. 147, No. 6, December 2000

scheme. This can only be made possible by visual inspec- tion of the error distribution when the errors are presented in an image. Fig. 8 illustrate such a visual comparison. The original ‘Lena’ image is given in Fig. 8a to provide a benchmark for observation of those noticeable errors, and the two error images, produced by JPEG-LS and the proposed scheme, are given in Figs. 8b and c, respectively. From these illustrations, it can be observed that the difference between the two error images is indeed notice- able and significant. In addition, it can also be noticed that the distribution of these large errors supports our analysis given in the last section. Namely, most of the large errors are produced around the area where diagonal edges exist, as evidenced by the original Lena. In Fig. 8c, it can be further observed that noticeable errors still exist around areas with diagonal edges. These illustrate that it is not possible to eliminate all the predictive errors purely on the basis of information given by those encoded pixels. Compared with other areas where errors are hardly notice- able, however, potential still exists for further research to improve the prediction around the diagonal edge areas.

In comparison with JPEG-LS, it can be established that the proposed scheme also reserves a similar level of computational cost and algorithm complexity. This can be evidenced by the flow chart given in Fig. 7. Specifically, two more condition testings are added to the original JPEG-LS prediction scheme, which produce two additional prediction values. Other than that, everything else is maintained the same.

b

a

C

Fig. 8 a Onginal ‘Lena’ b Predictive errors In ‘Lena’ produced by JPEG-LS c Predictive errors In ‘Lena’ produced by proposed scheme

fisual comparison of predictive error images

519

4 Conclusions

In this paper, we have presented our recent research work on the prediction scheme adopted by JPEG-LS, the latest international standard for losslesshear lossless still image compression. The work was prompted by the principle that predictive value should be constructed to maximise the exploitation of information given by those encoded pixels and predictive pattern to achieve the best possible accu- racy. On the basis of analysing the existing JPEG-LS prediction scheme, it is revealed that it is not sufficient to consider only horizontal edges and vertical edges in determining the predictive value. Although this design has covered most of the cases in practice and provided excel- lent predictions in applications, significant errors can still be noticed in those areas where true diagonal edges exist. To this end, a thorough re-analysis has been made of the JPEG-LS prediction scheme and two new predictive values have been added to compensate for and reduce the predic- tive errors in the diagonal edge areas. Experimental results support the proposed scheme, when the performance is measured by MSE values, which is a standard procedure to assess any prediction accuracy. In addition, the improve- ment in prediction accuracy is also illustrated by visual inspection of each individual error when all predictive errors are presented as an image. Further research work is also identified by the experimental analysis. This includes: revising the statistical modelling to investigate the possibility of improving the overall compression effi-

ciency (measured by compression ratios); and further investigation of the diagonal edge detection to promote its performance so that the error image is as dark as that in areas where vertical and horizontal edges are detected.

5 Acknowledgments

The authors wish to acknowledge the funding support from the Chinese Academy of Science, Beijing, People’s Repub- lic of China.

6 References

1 ISO/IEC JTC I/SC 29/WG1 FCD 14495-public draft: 1997 JPEG-LS: lossless and near-lossless coding of continuous tone still images

2 ISO/TEC Intemational Standard 10018 Part 1 and 2: 1994 Digital compression and coding of continuous-tone still images

3 http://www.jpeq.org/public/jpeqlinks.htm 4 RICE, R.F.: ’Some practical universal noiseless coding techniques: III’,

Jet Propulsion Laboratory, Pasadena, CA, Technical report JPL. 91 -3, 1991 JIANG, J.: ‘A low cost content adaptive and rate controllable near lossless image codec in DPCM domain’, IEEE Tiuns. Image Process. (to be published)

6 JIANG, J., and REDDY, M.: ‘An open-loop rate control scheme for JPEG-LS near lossless image compression’, Electron. Lett., 1999, 35, (6), pp. 465466 WEINBERGER, M.J., RISSANEN, J.J., and ARPS, R.B.: ‘Applications of universal context modeling to lossless compression of grey-level images’, ZEEE Trans. Image Process., 1996, 5, (4), pp. 575-586 WEINBERGER, M.J., RISSANEN, J.. and ARPS, R.: ‘LOCO-I: A low complexity, context-based lossless image compression algorithm’. Proceedings of Datu Compression Conference, Utah, 1996, pp. 140-149

5

7

8

580 IEE Proc.-Vis. h u g e Signal Process., K I ~ . 147, No. 6, December 2000