paper no: 200000 - college of engineering - purdue …mohtar/iet2007/073083.doc · web view2007...

Send your completed paper to Sandy Rutter at [email protected] by 13 April 2007 to be included in the ASABE Online Technical Library.

If you can't use this Word document and you'd like a PDF cover sheet please contact Sandy.

Please have Word's AutoFormat features turned OFF and do not include live hyperlinks. Your paper should be no longer than 12 pages. For general information on writing style, please see http://www.asabe.org/pubs/authguide.html.

This page is for online indexing purposes and should not be included in your printed version.

Author(s)

First Name Middle Name Surname Role Email

Michio Kise ASABE Member

[email protected]

Affiliation

Organization Address Country

United States Department of Agriculture, Agricultural Research Service

950 College Station Road, Athens, GA

USA

Author(s) – repeat Author and Affiliation boxes as needed--


Bosoon Park ASABE Member [email protected]

Affiliation




USA

Author(s)


Kurt C. Lawrence ASABE Member

[email protected]

The authors are solely responsible for the content of this technical presentation. The technical presentation does not necessarily reflect the official position of the American Society of Agricultural and Biological Engineers (ASABE), and its printing and distribution does not constitute an endorsement of views which may be expressed. Technical presentations are not subject to the formal peer review process by ASABE editorial committees; therefore, they are not to be presented as refereed publications. Citation of this work should state that it is from an ASABE meeting paper. EXAMPLE: Author's Last Name, Initials. 2007. Title of Presentation. ASABE Paper No. 07xxxx. St. Joseph, Mich.: ASABE. For information about securing permission to reprint or reproduce a technical presentation, please contact ASABE at [email protected] or 269-429-0300 (2950 Niles Road, St. Joseph, MI 49085-9659 USA).

mailto:[email protected]






http://www.asabe.org/pubs/authguide.html


Affiliation




USA

Author(s)


William R. Windham ASABE Member

[email protected]

Affiliation




USA

Publication Information

Pub ID Pub Date

073083 2007 ASABE Annual Meeting Paper




An ASABE Meeting Presentation

Paper Number: 073083

A Compact Multispectral Imaging System for Online Poultry Contaminant Inspection

Michio KiseUSDA-ARS, 950 College Station Rd. Athens, GA 30605 USA, [email protected]

Bosoon ParkUSDA-ARS, [email protected]

Kurt C. LawrenceUSDA-ARS, [email protected]

William R. WindhamUSDA-ARS, [email protected]

Written for presentation at the2007 ASABE Annual International Meeting

Sponsored by ASABEMinneapolis Convention Center

Minneapolis, Minnesota17 - 20 June 2007

Abstract. The final goal of this research was to design and fabricate a compact, cost effective multispectral instrument for real-time contaminant detection at poultry processing plants. The prototype system developed in this research was a dual-band spectral imaging system that acquired two different spectral images simultaneously. It was a two-port imaging system that consisted of two identical monochrome cameras, optical system and two interchangeable optical filters. A spectral reflectance from an object was collimated by lenses and split identically in two directions by a beamsplitter, and then each light was focused on the sensor by lenses through an optical filter. Two optical filters, that determined the spectral characteristic of the imaging system, were interchangeable without complicated manufacturing process. To create an accurately registered two-band image, an image calibration algorithm that corrected lens distortions and lens-sensor geometric misalignments were developed.

The prototype imaging system and the image calibration algorithm were tested to evaluate the registration accuracy of the two-band image. The test showed that the imaging system could provide a two-band image of 3D object with less than one pixel registration error. The prototype system was also tested with a poultry carcass and the preliminary results showed that it could effectively detect feces and ingesta on the surface of poultry carcass.

Keywords. Keywords: Poultry contaminant detection, Machine vision, Multispectral imaging, Optical system design






IntroductionFood safety in the poultry industry is ongoing problem. Several deaths occur each year from public consumption of contaminated poultry. Potential contamination can occur when feces or ingesta is deposited on the surface of the carcass. Identification and separation of the birds contaminated by feces is critical to protect the consumer from a potential source of food poisoning.

Multispectral imaging, in which two to about ten different spectral bands image is obtained and hyperspectral imaging that measures even more than ten spectral bands have been used for contaminant detection for poultry carcass (Park et al., 2004). Park et al. (2006) have conducted hyperspectral image analysis on poultry carcasses contaminated by the feces and found that two spectral bands were useful to segregate fecal contaminants on the poultry carcass. According to their studies, a ratio of the specific two spectral bands (565 nm / 517 nm) can be an indicator of the presence of fecal contaminants on the poultry carcass as higher ratio suggests greater chance of being contaminants.

In this paper, a dual-band spectral imaging system that can acquire two spectral band images was developed for poultry contaminant detection. By incorporating an interchangeable filter design, imaging system can measure any two spectral bands without complicated manufacturing process. This paper focuses on the development of the prototype imaging system along with an image calibration algorithm, and an evaluation of the developed imaging system on the image registration accuracy. The developed imaging system is also tested for poultry contaminant detection using real poultry carcasses and contaminants.

MATERIALS AND METHODS

Optical System Design

The dual-band spectral imaging system developed in this research is a two-port camera system that consisted of two identical monochrome CCD cameras (EC1380, Procilica, British Columbia, Canada), an optical system (Edmund Optics, Barrington, NJ), and two narrow bandpass filters, as shown in Figure 1. To develop the system cost-effectively, the entire system was designed with off-the-shelf products and assembled manually. The optical components of the imaging system consisted of a front lens unit, a beamsplitter, two bandpass filters and two back lens units. A light reflected at an object was collected and collimated by the front lens unit, which consisted of two lenses with a 25 mm diameter, and then split into two ways by a beamsplitter as 50% of the light was reflected at right angle and another 50% was transmitted straight. Two bandpass filters were enclosed in C-mount filter holders and attached at the each exit port of the beamsplitter. This design would allow the imaging system interchange the filters without complicated manufacturing process. This is a great advantage in terms of a flexibility of the spectral bands selection, as compared to conventional multispectral imaging systems that integrate filters and sensors in one unit. Two narrow bandpass interference filters (Omega Optical, Inc, Brattleboro, VT) that had central wavelengths (CWL) at 520 nm and 560 nm, respectively, with 10 nm FWHM, were implemented for poultry contaminant detection. An identical back lens unit, which consisted of two positive achromatic lenses with 12.5 mm diameter and an aperture stop, focused an image on the CCD sensor of the each camera. The back lens unit was capsulated by a helicoid barrel that allowed the lenses to travel up to 7.5 mm along the optical axis for adjusting a focus. A fixed aperture (4 mm diameter) was placed in between the two lenses of the back lens unit.

2

Optical Axis

Front Lens UnitBeamsplitter Bandpass

Filter Back Lens Unit

Camera (Port 1)

CCD

Camera (Port 2)

CWL:560 nm

CWL:

520 n

m

106 mm

Aperture StopPositive Achromat

Negative Achromator

Double Concave

Positive Achromat

Figure 1. The dual-band spectral imaging system

The focal length of the imaging system could be configured by combination of the two lenses of the front lens unit. Therefore, the imaging system could have various focal lengths by using different front lens units without disassembling the whole system. Two types of lens configurations were prepared for this research. One lens unit, that consisted of 500 mm positive achromatic lens and -50 mm negative achromatic lens, had a magnification equivalent to a 15 mm focal length. With 400 mm achromatic lens and -25 mm double concave lens (DCV), another lens unit had a 10 mm focal length. The paraxial specifications of two lens units were summarized in the table 1. Both lens systems had similar f/number, which was quite large because of their small aperture stop. Compared to the 15 mm lens system, the 10 mm lens system had an advantage in terms of a size of field of view it could capture. However, it would be expected to develop severer distortion on the image because of the -25 mm DCV used in the lens system.

Table 1. Use the Table Caption Style above each table. Lens Type 15 mm 10 mmFocal Length 15.2 mm 9.8 mmField of view ±18.2 ° ±19.6°Working f/# 7.4 7.9Aperture Stop Radius 2.0 mm 2.0 mm

Image Calibration

The original image taken by the dual-band spectral imaging system suffered from an image misalignment. Figure 2 showed the images of a fixed frequency grid distortion target that had total 441 (21 ×21) dots with 10 mm spacing taken by the imaging system. The overlay image demonstrated that the dots in two images did not overlay at all. A lack of precision on the lens-camera manufacturing contributed geometrical misalignment between two image coordinates, which was critical to the fecal contamination detection algorithm based on the ratio of two images. The CCD sensor itself had a tolerance of ±0.15 mm on the position of the active image area relative to sensor package. Combined with a tolerance on the position of the sensor relative to the camera, the total position tolerance was about ± 0.25 mm. Considering the size of the pixel of the sensor (6.45 m), the sensor positioning tolerance would cause about ± 39 pixels offset between images taken by two different cameras.

3

Figure 2. Images of grid distortion target; (left) Image taken by the port 1 of the imaging system

with 15 mm lens; (center) Overlay image of port 1 with the image of port 2 taken with 15 mm lens; (right) Image taken by the port 1 of the imaging system with 10 mm lens.

The lens system also contributed the coordinate misalignment. Because the incident lights reflected at the same point went through different optical path after the beamsplitter split the lights in two ways, even though their optical designs were same, some lens manufacturing tolerances had to be taken into account (Fischer & Tadic-Galeb, 2000). Furthermore, imperfect lenses and lens placement created a nonlinear deformation on the image that was contaminated by a lens decentering and radial distortion. The right image of the figure 3 is a same distortion target taken by the imaging system with 10 mm lens system. A severe negative (barrel) distortion was appeared on the image. Lens distortion was especially significant for those optical systems that had a wide field-of-view associated with a thick lens, which, in case of the 10 mm lens of this imaging system, the -25 mm DCV of the front lens unit was responsible for this severe distortion.

The camera calibration process in this research was two steps: First, correct a lens-oriented error by applying a mathematical lens distortion model; Second, correct a sensor positioning error by projecting one image onto another image based on a pinhole projection.

A lens distortion of the image was corrected by applying a mathematical lens distortion model to the distorted image. It is known that the most dominant factor contributing to the distortion is the radial component, especially the first term plays significant roll. In this research, two-order distortion model proposed by Zhang (1998) was used for the lens distortion correction.

Let (ui, vi) be the ideal, undistorted image coordinates, and (ud, vd) the observed, distorted image coordinates.

Radial lens distortion is modeled by:

, (1)

where, k1 and k2 are coefficients of the radial distortion, (u0, v0) is the center of the radial distortion. (xi, yi) is the undistorted normalized coordinates, and is expressed as;

(2)

where, fx and fy are the scale factors in image u and v axes, and fxy the parameter representing the skew of the two axes.

4

The seven lens distortion parameters, k1, k2, u0, v0, fx, fy, fxy were calculated by Zhang’s method. More than two images of known planar pattern, such as a grid distortion target shown in figure 3, from different viewpoints were needed for the calibration. In this research, the distortion target shown in figure 3 was used as a target and placed on an optical table in ten different angles. As the result, the parameters shown in table 2 were obtained. Because the first term of the distortion function is the dominant factor of the radial distortion, the degree of distortion that the each lens would introduce on the image could be roughly known from the parameter k1. With having 10 mm lens system much larger k1 than 15 mm lens system, the result of lens distortion calibration test confirmed that the 10 mm lens system could cause severer distortion on the image than 15 mm lens system.

Table 2. Lens calibration parameter Lens Type 15 mm 10 mmPort Port 1 Port 2 Port 1 Port 2k1 -0.132666 -0.135951 -0.808473 -0.810012k2 -0.365543 -0.441955 0.630070 0.588400u0 328.707 310.952 328.345 313.319v0 258.549 257.294 252.794 238.446fx 1133.23 1126.23 983.895 980.946fy 1133.79 1127.62 983.961 981.615fxy -0.135123 -0.261304 -0.035496 -0.047168

As the result of the lens distortion correction, the unique distortion on the original image was eliminated and the new undistorted image was subject to the ideal pinhole camera geometry in which a 2D image of a 3D object was formed by perspective projection. If the optical axes of two cameras are correctly aligned by the lens distortion correction and the two sensors are in parallel each other, the geometric relationship of the two images can be described by 2D linear projective transformation. Assume a given point in image frame 1, m1=(x1,y1)T, presenting a same object image frame 2, m2=(x2,y2)T. The geometric relationship of those two points can be described by the following equation:

(3)

With some known corresponding points between two images, the projection matrix of the equation (3) can be calculated by a least squares method. In this research, the ten images used for the lens distortion correction were used to calculate the projection matrix, and as the result, following matrix was obtained:

,

where, H1 and H2 are projection matrices for the 15 mm lens and for 10 mm lens, respectively.

Results and Discussions

Evaluation on Image Registration Accuracy

Static tests were conducted to evaluate an image registration accuracy of the two lens systems. With the imaging system fixing on the optical table, the distortion target was placed

5

perpendicular to the optical axis of the imaging system at six different distances with 5 cm separation. A two-band image was taken by the imaging system at each distance, and then the calibration method and parameters described in previous section were applied to the image. The locations of the center of the dots in the images were detected by an image processing and the offset of corresponding points between two images were calculated as the image registration error.

Figure 3 demonstrated that a two-band image taken with 15 mm lens system at 50 cm was registered accurately as the result of the image calibration. A composite image (left) showed that the misalignment of the two images appeared on the figure 2 was corrected, in which all dots were accurately aligned over two images. The bubble chart (right) showed a spatial distribution of the registration error over the image with a size of bubble at each location indicating the registration error at the corresponding dot in the composite image. It showed that the registration accuracy was fairly uniform over the image, and of total 441 dots, only 23 of them had a registration error larger than one pixel. The average registration error over the image was 0.63 pixels, and the maximum error was 1.13 pixels.

Error=1.0 [pixel]

Figure 3. Result of image registration with 15 mm lens system; (left) Overlay image; (right) Spatial distribution of the registration error with diameter of dot indicating size of registration

error.

The test results with 15 mm lens system were summarized in table 3. Overall, very similar results were obtained at all distances tested, at 50 cm, 55 cm, 60 cm, 65 cm, 70 cm, and 75 cm. Because the target object for this research was a chicken carcass, it was very important to know that the algorithm could perform image registration on 3D objects consistently. As for the 15 mm lens system, the result proved that the dual-band imaging system with the image calibration method could provide registered band image of 3D objects with less than a pixel error.

Table 3. Image registration error 15 mm lens system 10 mm lens systemDistance [cm] Average

[pixel]Maximum [pixel]

Distance [cm] Average [pixel]

Maximum [pixel]

50 0.63 1.13 40 1.42 5.1255 0.62 1.32 45 1.79 6.9560 0.56 1.57 50 1.47 6.0365 0.63 1.19 55 1.43 6.0670 0.63 1.26 60 1.30 4.6375 0.66 1.13 65 1.26 4.72

6

The result of the test with 10 mm lens system was also summarized in table 3. Again, consistent results were obtained from the all images. However, the registration accuracy was much worse than one of 15 mm lens system. Prominent registration error was appeared around the perimeter of the image. The image registration algorithm in this research was designed for the undistorted images that could be modeled by a pinhole projection. Because the amount of radial distortion on the original image increased radially around the lens center on the image, the large registration error found at the corners of the image must be a result of imperfect lens distortion correction. The static tests with two lens systems confirmed that an accurate lens distortion correction was crucial for good image registration accuracy.

Fecal Detection Accuracy

The imaging system was also tested with chicken carcasses contaminated by the feces (duodenum, cecum, colon) and ingesta. The carcass was hung on a shackle with facing to the imaging system with approximately 50 cm separation. Fecal samples were manually introduced on the carcass surface at several locations. Two-band images were acquired by the imaging system with 15 mm lens system and examined that how reliable the fecal contamination could be detected.

Contaminants T=1.00 T=1.08Figure 4 Result of the contaminant detection; Four type of contaminants, duodenum, cecum,

colon, and ingesta were manually deposited on the carcass surface; Two thresholds, 1.00 and 1.08 were applied to the two-band image; Black pixel in the images suggested contamination detected by each threshold; Black circle in the image of T=1.00 suggested the location where

large false-positive errors were found.

Figure 4 showed one of the images of a chicken carcass taken by the imaging system with 15 mm lens system and the detection results based on the band-ratio algorithm. Four types of contaminant materials, colon, cecum, duodenum, and ingesta were manually deposited on the chicken carcass at the breast area. The ratio of the two images (560 nm / 520 nm) was calculated after the background masks were applied to the two images. Pixels less than certain reflectance (2.5% in this specific case) were masked out as the background, and then, fecal contaminants were detected by applying a threshold to the ratio of two images. Two thresholds (T=1.00 and 1.08) were tested: Black pixels in the images indicated contamination detected based on the two thresholds, where the ratio of the two images was greater than the threshold. Overall, the result demonstrated that the four contaminated spots were successfully

7

CecumDuodenum

ColonIngesta

discriminated from the skin. However, massive false-positive errors were observed in the T=1.00 image. The false-positives were improved as greater threshold (T=1.08) was applied. However, the T=1.08 failed to detect some contaminants, especially duodenum, which could be detected by the T=1.00. To improve overall detection accuracy with less false-positive, it is important to improve detection accuracy on duodenum. Because only one set of optical filters, 520 nm and 560 nm, was tested this time, other filter combinations should be tested to determine if they could enhance the difference between fecal materials and skin better than the current filter combination.

ConclusionA prototype dual-band spectral imaging system was developed. The imaging system could acquire two images in any spectral bands simultaneously by utilizing two narrow bandpass filters, two monochrome cameras, and optical components. Two types of lens system, a 10 mm focal length and a 15 mm focal length, were designed. The 10 mm lens system had an advantage in terms of a size of field of view, but also had severer lens distortion. A two-step image registration algorithm including lens distortion correction and image projection was proposed. With 15 mm lens system, the dual-band imaging system with the image calibration method could provide accurate registered band image of 3D objects with less than one pixel registration error. However for the 10 mm lens system, severe distortion developed on the image prevented the imaging system and image calibration algorithm to register images as accurate as the 15 mm lens system. It showed that an accurate lens distortion correction, especially radial distortion correction was crucial to achieve precision image registration.

The imaging system, the image calibration and contaminant detection algorithms were tested with chicken carcasses contaminated by feces and ingesta. Based on the two-band image (520 nm and 560 nm) acquired by the imaging system with 15 mm lens system, simple band-ratio algorithm could detect the fecal contaminants on the chicken carcass. However, the prototype imaging system would need further development at the spectral configuration of the optical filter to improve the fecal contaminant detection accuracy.

Acknowledgements

The authors would like to thank Alan Savage, Engineering Technician, Jerry Heitschmidt, Imaging Specialist, and Peggy Feldner, Food Technologist for their assistance in developing the system and conducting the experiments.

ReferencesFischer, R. E. and B. Tadic-Galeb. 2000. Optic system design, McGraw-Hill, New York, NYPark, B., K. C. Lawrence, W. R. Windham, D. P. Smith. 2004. Multispectral imaging system for

fecal and ingesta detection on poultry carcasses. Journal of Food Process Engineering, 27(5):311-327

Park, B., Lawrence, K.C., Windham, W.R., D.P. Smith. 2006. Performance of hyperspectral imaging system for poultry surface contaminant detection. Journal of Food Engineering, 75(3): 340-348

Zhang, Z. 2000. A flexible new technique for camera calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11): 1330-1334

8

paper no: 200000 - college of engineering - purdue …mohtar/iet2007/073083.doc · web view2007...

Documents