A VERSATILE COLOUR SYSTEM CAPABLE OF FRUIT SORTING AND ACCURATE OBJECT CLASSIFICATION

Gary Kay and Gerhard de Jager

Department of Electrical and Electronic Engineering University of Cape Town, Rmdebosch 7700

of 256' or over 16 million available co~0urs.

Yellow

RED

1. INTRODUCTION The first section in this paper describes the theory behind choosing a colour system for 011 liae fruit imagcpmc- essing.Thesystemhstobeversatdeeaoughtocope with a wide m g e of fruit colours. The system ch- is based on Hue. Satunti011 .nd hhsity @SI) colour systsms M t with in [l]. [2], a d [3].

The next section identifies the best fertures of the HSI system, inordato.ccuntaly sort fruit inosshorta time as possible.

The h a i section highl~gh how the HSI features of

The claws formed from several fruit sunpls allow for classification and sorting of fruit.

tbe image pixels cm be classified using cluster analysis.

2. CHOOSING THE COLOUR SY- The Red, Gism. Blue (RGB) colour system is used to display images 011 a colour monitor. The RGB cube shown in figure 1. rspresmts all possible colours available for display on the RGB monitor. Each primary colour is &up of 256 grey level intensities. This mslrts a total

For the plrpaws of cxperimeatation the following equipmalt m u s e d iaordcr to display .ndpmcea fnrit iImgC6:

M RGB output cu~ltlp, the MVP-AT colour fnmc gnbbsr, md M IBM compatible 386 16MHzmicmproccssor.

I t m rolmdthatclassifying fruit .ccording to thcir RGB pixel componmts m computatidy expeasive. due. to the large number of combidions of RGB bteamitiies in a pixel. Ham M alternative morc prowsable colour system was sought after. The first alternative colour systems looked &were :

- the x,y,z modbate 1931 chrompticity cbrrt, - the X,Y.Z modbate tristimulus colour values, - the L.u,v coordlllltc . 1960 CLE uniform Chromaticity M e (UCS). - the U*.V*.W* modhate 1964 CIE system W.

145

TH0482-0/92/0000.0145 $1.00 01992 IEEE

These systems were all capable of converting the RGB coodbks respectively. However, it was only the HSI system that showed promise as M alternative colour system in a computing and industrial mvironmmt. as we11 as in psychologicpl visual represeatotion.

The HSI system involves a (3x3) mrttix multiplication to the (3x1) RGB column vector. This rcsult yields the (3x1) (I.vI,v2) column vector. whm I is itmsity and vl, v2 are line vectors which make a Hue angle and Satuxation radius. The resulting transformation effectively rotates the RGB cubc of figure 1. so that a two d immsid hexagon forms. The H-S plane can be viewed by looking along the axis from the white to the black COmcT of the cube.

Hence in this mtated COnfiguRton HSI can be defined as fouows: I) The Line of Varying Intensity :- line from black

towhitecorner. S) The Line of haeasing s.tuntion :- radial line

from the centre outwrrds on the fla colour hexagonal HS plane. The Angle of Varying Hues :- anticlockwise angles from the line from I to a primary colour (gg Blue).

H)

The colour surfna of images can be morc easily lmdnstood in terms of HSI regions rather than RGB pixel values in the RGB cube. There arc wed ways of

since them is no hkmationalstandprd. Eachtransfomationmatrix is

R,G,B in mind. If the curves seem equally weighted then

the hexagon is irregular. The transformntion is also de&@ with a particular colour hue as the 0 depee starting angle. a Blue).

writing the t m l h m a b 'on rmtrix

&ped with particular spectnl reflectance curves of

P mgular hexpgolr W form on the H-S PI-. othmvise

Of the several mnsfomations experimeated with, m e such trpnsfodon proved effective for apple sorting (equations 1 and 2). The transformation was based on those given in [I]. [2] & [3] but geomstricpUy modified so that 00 represcntf Blue, 120' rep-& Grren and 24$ repmeats Red.

With the above RGB to HSI transformation it was W b l e to simulate, in software. a modified version of P RGB to HSI converter. In this HSI system all fruit rpnging from bluish gnen to purplish rad could lie betwear the hue angles of 52' and 308' respectively. This urans that 360 lev& of hue can be mppped to 256 levels,

also have U most 256 levels. Heace each pixel could have three fc;rtunsof H. Sand I uEBoci.tdd with it, with

and with aptions 1 and 2, SrtURtion and intensity can

e4.I rsMnebeing ow byte si?z - this 6limiMw floating point O p e n U i ~ and incrsuas classifidon speed.

3. CHOOSING TEE BILW FEATURES OF THE COLOUR SY!3l'EM TO SORT FRUIT

Om may now think &at if H.S and I cm each have 256 levels thar them is no repun why o m should hawform the R,G .ad B 256 level componcats. Howaver, the fundamma remm for the trnnsforrmtion is that now all colour pixels cm be rsprssented bkly accurately with one feature - Hue. The fepturss of intensity and Bpturation ue only I K C C B ~ U ~ if further .ccu~pcy is needed.

The foUowhg ex-tal results show which fsotures best mutribute to a fruit classifier.

3.1. Classifying with Hue Only The simpcst md most effective m y to classify the fruit colour is to use the hue hstogmn (figure 2a). The example of figure2 shows that the p e a apple with slight piuk blush has a hue over a certaiu m g e (162' to 2304. By defining hue threshold levels one can get the percentage distribution of certain hue ranges within a fruit image.

3.2. Chsification with Hue and Saturation The two dimsaeioarl pixel plot in figure. 2c shows how the HS values of the fruit pixels cluster 011 the H-S plane. In this case (as m e n t i O D e d earlier) hue angles are mcuwQd MticlockwiBe froan the borizona (blue) U i s . It has been fwnd that the sahuation componmt is mmdiately dcpmdmt on the type of lighting m the fruit.

146

2: a) the birtopnm reprrrm dI the pixel hues in the windowed fruit. b) the windowed fruit U i Gnrmy Smith with dight pink blurh. c) the cluncn of fruit pixeli on the H-S p b .

Hence. a yellow light (incan-t) form a tight cluster of green and red on a fruit, whereas a white light (daylight) tends to distinguish gree~ and d H-S clusters more readily.

This classification method is effective if a constant colour temperature lighting system is used. Only the hue value represents what the actual colour is. Hence saturation (amount of colour) and intensity (amount of whte in the colour) histograms alone cannot classify colour.

3.3. Classifying with Hue and Intensity It is found that blemishes on a fruit change more UI intensity than in saturation. For further muracy in fnut classification 4 blemish identification ) the H-I plane is used. Accuracy in classification generally meam that if features are distinct. and can be separated easily, then they can be classed easily. For example a dark area on a fruit is distinct from a light area on the fruit (using intensity), and a red area on a fruit is distinct from a p m area (using hue).

Figure 3c illustrates the clustering of the green apple with slight pink blush (figure 3b), on the H-I plene.

One problem in using the intensity feature is that it varies depending on the light positions. In order to gain the full benefit of the intensity over the fruit 4 only vary where

I I Figrt 3: a) the -m of dI pixel utudoas in the windowed fruit, b) the widowed fiuit U i Gnrmy Smith apple with pink b i d , c) the cluacring of pixels on &e 2D H-I p b .

blemishes occur) the lighting must be uniform over the fruit.

4. CLASSIFYING USING CLUSIER ANALYSIS

Various cluster analysis techniques have been investigated. The methods were to be used on the H-I clusters, since it was thought that clusters were more visually separable on the H-I plane as opposed to the H-S plnne. Classification was not made using d three feptures since computation time was not justified. The following are cluster analysis techniques invatigated:

-the supervised classification methods of the Bayesian Parametric classifier, and the minimum distance classification using Euclidean distances

-the unsupervised classification methods of the (51.

K - m algorithm, and ISODATA ([5],[6]).

It was found that the Statistical Analysis Software (SAS) available on the Ufl Vax, provided the above (and more) cluster analyzing methods. In particular the SAS function 'FASTCLUS' proved capable of analyzing the large data sets of fruit H-I pixels (71. The function uses & iteration to classify the pixel clusters. The function is based on the K-means, nearest neighbour algorithm (as in (51).

147

By using s e v d apple samples of varying blemish Sizes, theunsupervlssd . classification lod to, at moat. four distinct dam camids @ cluster maas). It is now possible to cm through the H-I pixels in the fruit and bin

d d ) . Atter the single cm a fraqusacy count for

in theclrss the apple a n becladialwordhgly.

thepix&intotbsir.ppmpnuc . clrss(theasMstclass

pixasin acb class is givm. Based on tbe frrqu4ncy size

161 14 62.5 171 64 10.0

The suptnrised method of finding the colour class d (or cluster mems) was perfonnsd by averagiag a d window (less the 25 pixels in area) for eacb colour region visually nprsseating a c h on the image. H a m the four class centroids for the Granny Smith with sight pink blush, wm obtrioed from regions: 1) on the light -put, 2) on the duk green put. 3) on the light brown pmt. 4) and on the Mc brown or middle of the blemish mgim on the fruit. colour segmmention using tbesc four c h and the H and1 feahues of each pixel gave the bsst miaimurn distawe classification of the colour pixels. It was found tlut from most fruit huge expwin~~ts (see Table I for an example), the class aatraids with H-I farturts 0bt.ined by supmissd CLssifiCrtion (mmuruy selecting clrsses) werc as good as thcseobEpinsdbyumupemd * classifidon (K-merns iteration to find clrssss). However, the unsupen)ised clasdication method gave a bttter colour segmeatation w h a ~ the pixels OII the H-I plane formed a tight cluster md msny colour clpsses (four) wtre difficult to visually diStinguish.

5. CONCLUSIONS

The HSI colour system has several dvcmt.ges over other

- .ccunte colour classificatim is poesibk with only one pmm#at, d y the hue vdm;

acumcy and blemish mating; - the timo to cbsify image using only hue is much h t e r (by order 3) than the RGB - a m m u b l y priced camater cm canvest RffB to HSI and harcc allow for d timB, rutormb 'c fruit sorting; -cl- d p i ~ the H-I pllas all- for to be segmeated, and brJlce the M. of colour ftrltprss cm be *al. - suparvissd clrssificltion allows colcm pixels to be sepmtcd just aswdl as u n q w v m d ' classification, if

colour systems, these ue:

-tbeintealsityaadsltuntimallow f a ! iu the fCl . s s i fy ing

clustsre ~ . l l bs v h d l y -lid tbe H-I p h .

6. AKNOWLEDGEMENTS

Thanks go to the employees of Tddogic in Soxmmet West. witlnmt whom this project d d not have bem initiated md to Teklogic for wxpport.

7. REmRENcEs

[l] Lehr A.F.. Stova3 R.J.,(1984): 'Highspeed

Inmgaz'.IEEE CGBtA, Fsbnrrry pp 34-39. [2] Niblack W. (1986): ' h g o Display'. An Intmduction

[3] Data Tnnsl.tion Mand. Data TrrnsLtion krc. [4] W y d G., Stiles W., (1967): colour &am,

[5] Niblack W. (1986): 'c1.ssifiCrticm'. An Introduction to Digital Image procasSine. Q7. pp 167-189. [6] Ball G., Hall D.. (1%5) : ISODATA, A Novel Method of Data Analysis and p.#em Classification.

SASlSTAT Users Guide (1990): 'The FASTCLUS

M.niprl.tion of the coiaur chrormticity of Digital

toDigit8lImrgepIocsgan .g.chz*pp23-67.

ppssim.

procedure'. 4th al.. V011, Ch22. pp 823-850.

148