672

INDUSTRIAL COLOUR INSPECTION BY VIDEO CAMERA

C Connolly and T W W Leung

University of Huddersfield, England

BACKGROUND

Colour is an important property of many mass produced items, since it attracts the eye and conveys information quickly. Some packaging colours are so closely identified with particular brands as to be trade marks, for example Coca-Cola red and Cadbury purple.

The colour of retail products and their packaging must be consistent over long production runs, and from one run to another. When colour consistency is poor, an impression of low quality is formed in the mind of the customer. This is especially damaging in food packaging, where poor appearance gives the impression of stale products. In many industries, colour assessment is carried out off-line by production staff who make a visual comparison between the current production and a standard. Sometimes spectrophotometers or colorimeters are used to give precise colour measurements; these instruments, generally, are also used off-line.

Recent advances in video camera technology, coupled with techniques developed at the University of Huddersfield, allow video cameras to be used as colour measuring instruments. Cameras are non-contact sensors, and can therefore be used on-line without disrupting production. On-line inspection enables a faster response to faults, saving waste and improving quality.

This paper describes the research done at the University of Huddersfield, presents the results of laboratory tests and addresses two particular industrial applications.

CHARACTERlSTICS OF VIDEO CAMERAS

Cameras have three major advantages over colorimeters and spectrophotometers for industrial colour inspection. Firstly, they can be used on-line. Secondly, three- dimensional objects may be inspected. Thirdly, many colours may be inspected simultaneously. In contrast, spectrophotometers and colorimeters are designed to measure a uniformly coloured flat surface in contact with the instrument's aperture, typically of 5 mm diameter.

However, cameras are not meant to be accurate colour measurement instruments. They are primarily designed to reproduce colour images of a quality acceptable to

Image Processing And Its Applications, 4-6 July 1995 Conference Publication No. 410,O IEE 1995.

the human eye. In order that video cameras may be used for colour measurement purposes, the principal requirement is that should be stable and their response repeatable. An additional requirement is that the camera should be sensitive, in order to detect small colour variations in the product.

Beginning in 1985, Connolly et al (1,2,3) investigated the sources of colour error in video cameras and in images of three-dimensional objects. This work has been continued more recently by Leung (4). A scientific study has been made of the errors inherent in video camcras, including noise, drift in gain and balance, saturation of small bright areas, the effect of background colour, and quantisation errors. Three different types of camera have been studied, and techniques have been developed to overcome or compensate for their errors. This work has culminated in the development of a method of automatic camera control, for which a patent has been filed by the University of Huddersfield (5).

EXPERIMENTAL STUDY OF CAMERA ERRORS

Three types of video camera have been tested:

(i) a Shibaden HV-40SK monochrome vidicon

(ii) a JVC BY-1 10E 3-tube saticon camera; (iii) a Hitachi HVC I O 3-chip CCD camera;

The tests involved capturing images, windowing areas of interest, and plotting pixel clusters in RGB other colour spaces. The procedure was described in (2) and (3).

When the scene comprises uniformly coloured flat matte surfaces under uniform illumination, the spread of each cluster results from the inherent noise in the camera. Cluster size is a measure of the precision which may be achieved by that particular camera. Cluster separation determines the closeness of the colours which can be distinguished. The position of the cluster is affected by the gain of the camera and its colour balance, and by the colour of the illumination. In a system where colours are taught and then inspected, any movement in the position of the cluster from one image capture to the next will cause colours to be wrongly classified.

camera, used with colour separation filters;

673

Monochrome Camera

Preliminary work with the monochrome camera made it clear that the auto gain control circuit creates problems in the accurate sensing of colour. If the gain is allowed to adjust itself independently through the three colour separation filters, this tends to desaturate the colours in the scene, and also leads to a variation of colour balance according to the dominant colour of the scene. This means that objects produce different signals if presented on different coloured backgrounds.

There was a tendency for small areas of bright colour to become saturated through one or more of the filters, because the auto gain control responds to the average signal, not to its peak. This effect has been noted by Klinker (6), who refers to it as "colour clipping". The camera signals were found to vary markedly with camera temperature.

To improve the repeatability of measurement with the monochrome camera, the auto gain circuitry was disabled, and the camera set by hand. To avoid colour clipping, the settings were adjusted so that a white object in the scene gave signals some way below the maximum detectable signal. The camera was repositioned well away from the lights, to avoid overheating.

Three-Tube Camera

Experiments were then carried out with the 3-tube colour camera. The red, green and blue channel analogue outputs were each digitised by 8-bit analogue to digital converters in a Matrox MVP-NP framestore card fitted into a personal computer. To test the precision and repeatability of the camera, test objects, in the form of flat mondrians, were viewed repeatedly over a period of time, and the data was analysed statistically.

The 3-tube camera was tested for repeatability over 5- hour and 1 I-day periods. The camera was preheated, then switched on and calibrated according to the standard user instructions. Then repeated measurements were made of the test object, without further calibration. The camera was adjusted manually to suitable settings, which then remained constant throughout the test period.

It was shown by these experiments that the RGB values varied with time, changing rapidly in the first half hour, and more slowly From then on, but never settling down completely. Tests again revealed that when the colour patches were displayed on a different background, the 3-tube camera gave different measurements. The RGB values of any colour patch were biased towards the colour of the background.

The time variation and the background effect may both be overcome by including in the scene a standard object, and by adjusting the camera to achieve the same readings on this standard every time an image is captured. This ensures repeatability. Any variations in light source or in camera gain are cancelled out.

This control technique gave a significant improvement in the stability and repeatability of measurement, as shown by figures published by Connolly et a1 (7). However, the control technique was time consuming and frustrating, since the operator was required to adjust the camera controls each time an image was captured, in accordance with advice issued by the computer.

Three-Chip Camera

With the 3-chip camera the situation was improved in two ways: firstly, the solid state technology is less noisy than CRT technology, so the cluster size is reduced; secondly, the camera has an RS232C remote control link. which allows direct control to be exercised by the software. The camera settings may be controlled each time an image is captured, ensuring repeatable readings, while freeing the operator from the need to make any manual adjustments. The camera control techniques developed in the studies with the monochrome and three-tube cameras were implemented in a special control program (4), for which a patent has been tiled ( 5 ) . This software controls the aperture, the gain, and the colour balance of the 3-chip camera.

SUITABILITY O F VARIOUS COLOUR TRANSFORMATION EQUATIONS

As has already been mentioned, the size and separation of the pixel clusters affects the precision of colour measurement and the power of resolution of close colours. When scenes involve three-dimensional objects, their surface colours appear to vary because of lighting effects, some areas being highlighted, and some in shadow. This leads to larger clusters. By transforming the pixels' RGB values to other colour spaces, the size and shape of the clusters are altered. A study of various colour space transformations has been carried out to identify which produce the smallest and best separated clusters for three-dimensional scenes.

Another important consideration is the uniformity of each colour space. Uniform colour spaces have been routinely used with spectrophotometers and colorimeters for many years to provide a simple framework for deciding between acceptable and unacceptable colour differences. They have been developed over a period of many years from studies of flat two-dimensional objects, and it is only recently, for example by 'Tominaga (8), that they have been applied in the field of machine vision. A uniform colour space

674

aims to transform instrumental measurements into metrics proportional to perceptual colour differences.

Nine different colour spaces were studied to find which was the most suitable for use with images of industrial scenes. Full details are published in (3) and in Connolly (9), but a summary of the findings is given here.

Clusters from 3-d objects

Recent work in physics-based vision shows that, under certain conditions, the light reflected by a three- dimensional object with uniform surface colour has constant spectral balance, but varies in intensity. H.C. Lee et al (10) provided evidence to support this from tele-spectroradiometer measurements of the reflectances of various three-dimensional objects. Klinker (6) developed an algorithm to segment scenes whilst ignoring colour changes due to highlights and shading. Klinker worked in RGB space, but a simpler approach is to transform the RGB values to a colour space in which the chromaticity metrics are independent of light intensity. The pixels from a uniformly coloured surface then form a small cluster, and segmentation becomes simply a matter of thresholding either side of the cluster.

The nine colour transformations studied included the following from classical colour theory, colour television and machine vision approaches:

CIE 193 I chromaticity diagram x,y; CIE 1960 uniform chromaticity u,v; CIE 1976 L* U* v*; CIE 1976 L* a* b* (CIELAB); PAL L U V; computer graphics HSV; Yuichi Ohta I1,I2,I3.

The equations recommended by the C.I.E. are published in many textbooks, for example Wyszecki ( I 1). The PAL LUV equation may be found in Patchett (12), the computer graphics algorithm in Heam and Baker (13) and the final set of equations in Ohta (14).

Pieces of brightly coloured matte paper were cut, rolled into cylinders, and stuck on to black card. Eight colours were used: red, orange, yellow, green, blue, violet, brown and white. The cylinders, their axes horizontal, were illuminated from above by five D65 fluorescent tubes, and the 3-tube saticon camera was placed in front of them. The geometry was designed to avoid specular reflections but produce shadows. A vertical strip of pixels from each cylinder was studied, using each of the above transformations. If a colour transformation is to be useful for the inspection of surface colour in three- dimensional objects, the pixels from each colour must form a tight cluster in the chromaticity plane, clearly

separate from the other clusters. The clusters were separate in xy, u*v* and a*b* chromaticity planes; in the other spaces, the brown and orange clusters touched.

Uniformity of Colour Spaces

The Munsell Book of Color (15) provides a set of physical samples arranged in equal perceptual colour steps, and has been used by many researchers (1 1) for testing the uniformity of colour spaces. A selection of chips from the Munsell Book was plotted in each colour space, to compare their uniformity. Tabulated colour measurement values from Wyszecki and Stiles (16) were used in these calculations.

Details of the uniformity results were given in (3), and CIELAB emerged as the most uniform of the colour spaces studied. Since CIELAB gave good cluster separation [or the three dimensional cylinders and is already widely accepted throughout industry for the measurement of colour differences, we chose to use it in our colour inspection system. However, it is shown in (9) that the chromaticity metrics of CIELAB are intensity dependent. Further research is underway into the use of a logarithmic colour space, in which the chromaticity metrics are independent of illumination intensity. The uniformity of logarithmic space is better than CIELAB in its hue metric, but slightly worse in chroma.

PERFORMANCE AND APPLICATIONS

The 3-chip camera, controlled by our special software, was tested for repeatability. The standard pattem was a white card displaying 17 coloured rectangles from the Pantone Color Formula Guide (17). The worst colour difference recorded was 1.61 CIELAB units, and the average was 0.33 units. Details are given in (7)

The repeatability of our camera-based inspection system is an order of magnitude worse than that of a typical high quality benchtop colorimeter or spectrophotometer. Nevertheless, the advantages of the video camera make it a suitable instrument for on-line colour inspection in a variety of manufacturing industries. The camera-based colour inspection system has been demonstrated to potential users in the fields of textiles, construction, printing, brewing, and beverage can production.

Textile Yarns

One particular application in the textile industry involves the inspection of yarns. The standard yam, a sample which was slightly off shade but acceptable, plus a sample which was unacceptable in shade, were studied. The three yams were attached to a sheet of white cardboard which was propped up in the back of

675

the lighting cabinet with five fluorescent tubes lit. The standard colour was trained, and the other two samples were used to set acceptability limits. Then the three yams were repeatedly measured over a period of 9 days, without retraining the system. The samples were kept in an envelope between inspections to avoid fading.

The average colour difference between the original measurement of the standard yam and subsequent measurements was 0.96 CIELAB units. This is higher than the 0.33 CIELAB units observed for flat objects, because of the disturbance caused when the yam is taken out of the envelope, and the fact that its shape incorporates some shadows. However, the repeatability was sufficiently good for the colour inspection system to correctly label the acceptable and unacceptable samples in all 26 sets of measurements over a 9-day period.

Beverage Cans

Another application is the inspection of beverage cans. On some production lines, the cans are filled before they are printed, so waste due to unacceptable colour is expensive. Current practice is to remove a can from the line every 15 minutes, and inspect it either by eye, or with the help of a hand-held spectrophotometer, using its smallest aperture so as to minimise the effect of the can's curvature. Since the cans are produced at the rate of approximately 1000 per minute, waste rapidly accumulates once the colour goes off-shade.

Beverage cans are a very challenging application, since the objects are three-dimensional, highly reflective, and of high-chroma colours. Our first measurements were disappointing. Table 1 shows measurements of a pair of soft drink cans - the standard colour 'std' and an off- shade sample 'samp'. The cans were measured by a spectrophotometer, and by our automatically controlled 3-chip camera, using its normal setting of 0dB gain. The camera measures a colour difference "Col. Diff." between the standard and sample cans, but in the opposite direction from the spectrophotometer as regards the b* metric. Note that we are not concerned with the absolute accuracy of measurement, but with the accuracy of the colour difference between sample and standard.

The problem is due to the quantisation errors of the 8- bit analogue to digital converters. Both the G and B channel measurements of the standard can are equal to the black levels for these channels. The sensitivity of the human eye increases at low signal levels, and spectrophotometers mimic this behaviour. The camera, however, uses linear analogue to digital conversion, so the quantisation errors become very significant at low signals. lkeda et al (18) showed that the worst colour difference between the original analogue values and the

digitised RGB values using 8-bit analogue to digital converters, was 10 CIELAB units. In order to reduce this to 1 C'IELAB unit, 12-bit analogue to digital converters ai-e required.

The use of 12-bit converters would add very significantly to the cost of the complete machine vision system; so advantage has been taken of the gain controls of the Hitachi HVC IO camera, which can operate at gains of 0, +9 and + I8 dB. The cans were remeasured with the camera at +18 dB gain. This provided extra resolution in the G and B channels, and a significant signal above the black level was detected in both channels. The amplified signal was scaled back to 0 dB by the following equation:

18 dB signal - I8 dB black 7.94

The R signal, of course, saturated with +I8 dB gain. Its 0 dB value was therefore used, after subtracting the 0 dB black signal. The CIELAB equation uses a non- linear function to calculate L* a* and b*, and it is affected by a change of origin. Subtraction of the black signal level leads to better agreement between spectrophotometer and camera measurements of colour difference. Table I shows that the camera was then able to detect the difference between the two cans, and was in agreement with the spectrophotometer about the direction of the colour difference.

0 dB signal :: ( 1 )

TABLE 1 - Colour Measurements of Beverage Cans

The technique developed for the inspection of beverage cans has been incorporated in our software control system. The software recognises when a signal requires increased resolution, and switches to a higher gain band when necessary. In this way, the equivalent of I I-bit resolution is achieved, with no increase in hardware cost. The charts in (18) indicate a worst case quantisation error of 1.5 CIELAB units with 1 I-bit digitisation.

The two can\ studied here were purchased from a retail outlet, so they illustrate a colour difference which passed the current quality control checks. The 'sample' can appeared faded in comparison with the 'standard, the colour difference being easily discemable to the eye. It is clear [hat our camera-based colour inspection system is able to detect this colour difference, and its on-line use would improve the quality of appearance of this product.

676

CONCLUSIONS

Cameras have three major advantages over colorimeters and spectrophotometers, making them much more practicable for industrial colour inspection:

a) they can be used on-line; b) three-dimensional objects may be inspected; c) many colours may be inspected simultaneously.

This paper has described research into the problems encountered when video cameras are employed for the inspection of colour in three-dimensional scenes. This has led to the development of an automatic control system which significantly improves the repeatability of the camera.

Trials to date on textile yarns and beverage cans have proved that this camera-based system is at least as good as current colour quality control systems. Moreover it offers advantages of reliability, reduced need for supervision, and immediate waming of colour drift.

ACKNOWLEDGEMENT

Leung's work was supported by a Bursary from the University of Huddersfield.

REFERENCES

1. Thomas W V and Connolly C, 1986, "Applications of Colour Processing in Optical Inspection", Proc. SPIE m, 116-122 2. Connolly C, Littlewood S and King E S, 1989, "A System for the Segmentation of Colour Images", IEE 3rd Int. Conf. on [maw Processing and its Applications, m, 44 1-444 3. Connolly C, 1990, "Image Segmentation from Colour Data for Industrial Applications", PhD Thesis, Polytechnic of Huddersfield, UK

4. Leung T W W, 1995, "High Accuracy Colour Segmentation of Industrial Images", PhD Thesis, University of Huddersfield, UK

5. UK Patent Application 94 16406.8, 1994, "Colour Inspection System", in the name of University of Huddersfield, Inventors: Connolly C and Leung T W W

6. Klinker G J, 1993, "A Physical Approach to Color Image Understanding", A.K. Peters, Wellesley, MA

7. Connolly C, Leung T W W and Nobbs J, 1995, "The Use of Video Cameras for Remote Colour Measurement", submitted to Joumal of Societv of Dvers and Colorists

8. Tominaga S, 1992, "Color Classification of Natural Color Images", Colour Research and &D . lication, 12, 4, 230-239

9. Connolly C, 1994, "The Relationship between Colour Metrics and the Appearance of Three Dimensional Coloured Objects", submitted to

h and -, . .

IO. Lee H C, Breneman E J and Schulte C, 1990, "Modeling Light Reflection for Computer Color Vision", IEEE Trans on Patte m Analvsis and Machine Intelli_eence, 12,402-409

11. Wyszecki, G., 1981, "Uniform Colour Spaces", Golden Jubilee of Colour in the C.I.E, Publ. Soc. Dyers and Colorists, Bradford, UK

12. Patchett, G.N., 1967, "Colour Television with particular reference to the PAL System", Norman Price, London, UK

13. Hearn, D. and Baker, M.P., 1986, "Computer Graphics" Prentice-Hall

14. Ohta, Y., 1985, "Knowledge-Based Interpretation of Outdoor Natural Scenes", Pitman, London, UK

15. "Munsell Book of Color", 1970, Munsell Color Company Inc, 2441 North Calvert Street, Baltimore, Maryland, 21218, USA

16. Wyszecki G. and Stiles W.S., 1967, "Color Science" Wiley, New York, USA

17. Pantone Inc, 55 Knickerbocker Road, Moonachie, New Jersey, 07074, USA

18. Ikeda H, Dai W, and Higaki Y, 1992, "A Study on Colorimetric Errors cause by Quantizing Chromaticity Information", nta i n Measurement Technologv Conf, 374-8