automatic visual measurement of surface-mount device placement

44 IEEE TR.4NSACTIONS ON ROBOTICS AND AUTOMATION. VOL 6. NO I , FEBRUARY 1 Y W

Automatic Visual Measurement of Surface-Mount Device Placement

Abstract- An experimental system for automatic visual measurement of surface-mount device placement on printed circuit boards has been developed. Infrared illumination is used to aid in segmentation of component lead and solder pad images from the background. Best fit lines through centroids of the lead and soldering pad images are calculated from which the displacement and angular orientation of the component relative to the pads is computed. Measurements are derived from a combination of slope and mean centroids from opposite sides of a device to reduce the effects of sensor resolution, missing pads, or missing leads. Since leads and pads are viewed simultaneously, the accuracy of the system is not dependent on precise positioning of the board within the inspection station.

The system accuracy has been tested with 50 commercially manufactured printed circuit boards containing 500 samples of J-leaded plastic leaded chip carriers (PLCC) and small outline integrated circuits (SOIC) of various sizes. The positions of the components were measured manually and are compared with measurements determined by the system.

I. INTRODUCTION

ONVENTIONALLY, electronic components have been C mounted on printed circuit boards by inserting terminals of the devices through holes on the board and applying solder between the lead and a pad on the opposite side from the component. More recently, surface mounting has become a popular technique for connecting electronic components to circuit boards. Components are soldered directly onto metal pads on the surface of the board without the use of through holes. Surface-mount devices are typically smaller than their dual-in-line (DIP) counterparts and weigh less [ 11. Smaller sizes together with the absence of holes for leads allow higher board density (with components mounted on both sides of the board) while at the same time increasing the area available for routing conductors in sublayers. Also, electronic speed is enhanced and power consumption is lower [2]. Popular surface- mount device packages include the plastic-leaded chip carrier (PLCC), the leadless ceramic chip carrier (LCCC), the small- outline integrated circuit (SOIC), and chip capacitors and re- sistors. For a comprehensive introduction to surface mounting technology, the reader is referred to [3]-[6].

Surface mounting has the added advantage that circuit boards are more suited to assembly using flexible automation, including high-speed pick-and-place mechanisms. How- ever, higher pin counts and smaller lead separations require

Manuscript received November 1 , 1988; revised July 14, 1989. This work was supported by the Natural Sciences and Engineering Research Council of Canada.

The authors are with the Department of Electrical and Computer Engineer- ing, McMaster University, Hamilton, Ont., Canada L8S 4L7.

IEEE Log Nuniber 8931963.

more precise component placement and board materials that can endure the thermal cycling of the vapor phase reflow soldering without warpage to ensure a flat mounting surface. Accurate placement of surface-mount components is critical in the soldering of high-quality circuit boards- even small misplacement may lead to soldering defects such as bridging and poor wetting. Moreover, a board containing improperly positioned devices may pass functional tests but fail after pro- longed use due to inferior solder joints. Without holes to guide component leads, however, placement accuracy is entirely dependent on the positioning of the circuit board and on the mounting mechanism.

The benefits of automatic visual inspection are well known in the manufacturing and assembly of circuit boards. Recent advances have been made in the inspection of bare circuit boards [7]-[ll] and solder joints [12]-[21]: however, relatively few systems have been demonstrated for the inspection of surface mounting.

A morphology-type processor which uses a custom-built image flow computer for measurement of chip resistor and capacitor placement is described by Buffa [22]. Placement of components is determined from examination of histograms extracted from the surface of solder pads. Accuracies of 1 mil in displacement and 1" in rotation are claimed. Chang [23] has developed a system which determines the position of a component from images of lead tips. A robot then places the device onto the circuit board. To determine orientation, a line fitting procedure is performed on a row of vertical lead tips and the center of the component is found from the lead tip centroids. This arrangement however relies on the accuracy of the robot; no inspection is done after components are loaded. More recently, Susuki et ui. [24] have described a system for loading gull-winged, flat-pack IC's onto circuit boards using a Cartesian coordinate type robot under visual guidance, Image processing techniques are used to determine positions and orientations of soldering pads which are matched to those measured for components to achieve accurate mounting. The system also detects bent leads.

This paper describes the design and performance of an experimental system for automatic placement measurement of surface-mount components with protruding leads. High-angle infrared illumination is used to obtain high-contrast images of metal leads and solder pads of components after mounting onto the circuit board. Placement and angular orientation are computed from the relative positions of best f i t lines through the centroids of pad and lead images from opposite sides of the component. Since the position of a device is measured rel-

1042-296X/90/0200-0044$01 .OO O 1990 IEEE

CAPSON AND TSANG VISUAL MEASUREMENT OF DEVICE PLACEMENT

infrared LED

lead solder

component

~ lead r sold:-,ead image I , I

component

1 lead r ~ o l d e r ,,,,,,-lead image

1 1 I - pad image

Fig. I . Leadipad image formation. (a) Lighting arrangement. (b) Typical images for J lead. (c) Typical images for gull wing.

ative to the position of its soldering pads in one camera view, the system does not rely on precise positioning provided by the mechanism used to move the circuit board to the inspection station. The technique is also tolerant of missing leads or soldering pads and images of the leadsipads corrupted due to lighting or sensor resolution effects. The accuracy of the system has been tested in the laboratory with commercially manufactured boards containing 500 samples of PLCC packages with “J” leads and SOIC devices with gull-wing type leads of various sizes. The results compare favorably with manu- facturer’s placement tolerance specifications and with manual measurements.

IT. DESCRIPTION OF PLACEMENT MEASUREMENT SYSTEM

A . Image Formation A special lighting fixture is used to illuminate the compo-

nent leads and solder pads. The light sources are plastic dome- shaped TRW OP290A infrared light-emitting diodes (LED’s) with physical dimensions of 5 mm (diameter) x 9 mm. Each LED has a maximum electrical power rating of 15 mW and a peak emission wavelength of 950 nm. Four rows of LED’s (one for each side of the components) are arranged in a square of 35 mm x 35 mm, positioned 22 mm above the PCB, and illuminate the board at an angle of 15” from the normal of the circuit board plane. Fig. l(a) shows a cross section of a “J” leaded device being illuminated. Due to the high illumination

(b) Fig. 2 . Photographs of yellow-colored SOIC. (a) Under visible light. (b)

Infrared illumination.

angle, only the surfaces of the tops of the leads and the solder pads can reflect light towards the camera. The “vertical” part of a lead and the solder joint fillet will not be detected from this view. Typical images obtained for “J” leaded and gull-winged type devices are shown in Fig. l(b) and (c).

The reflected light passes through a Kodak Wratten infrared filter No. 87. Infrared light is used for illumination because it is mostly absorbed by materials such as plastics and ceramics but reflected by metals. Since the copper traces on a PCB are covered by a solder mask, the metal leads and solder pads are usually the only exposed metals. The resulting image under infrared lighting contains bright regions of metal leads and solder pads in a relatively dark background, which are seg- mented using an automatically selected threshold. Fig. 2(a) shows the photograph of a yellow-colored SOIC package under visible lighting. The same chip under infrared light is shown in Fig. 2(b). Here, the image of the chip body is much less bright than under visible light, and segmentation is eas- ier. (In this example, only the bottom row of leads is being illuminated .)

The images are captured by a Javelin JE2062ER video camera with a resolution of 384(H) x 496(V) and sensitivity in the infrared region. A telephoto lens affixed to the camera is adjusted to give a maximum field of view of 40 mm x 32 mm. Standard RS-170 video from the camera is digitized to 512(H) x 480(V) with 7 b of gray level by a CORECO Oculus-200 frame grabber installed in an IBM PC-AT computer. The system software is written in a combination of assembly code and the C language.

B . Image Analysis Binary image techniques are used to process the pictures.

An automatic thresholding method described by Otsu [25] was used to separate pad and lead areas from the background. The threshold is taken as the average of the mean gray level of dark and bright regions in the picture and is determined with an interative procedure using the gray-level histogram.

4 6

lead image

IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION. VOL 6. NO I . FEBRUARY IWO

- 04:Ok

Boundaries of the pad and lead regions are run-length encoded and a fast connectivity analysis algorithm [26] is used to obtain the vertex points from which area and centroids are readily calculated.

For a correctly placed component, the resulting binary image will contain objects for each row of leads together with a corresponding row of solder pads. A typical example for an 18-pin PLCC is illustrated in Fig. 3. It is assumed that an XY positioning mechanism brings the circuit board beneath the sensor so that the soldering pads are approximately recti- linearly oriented within the image. For the two-sided SOIC devices, we assume that the leads are positioned approximately parallel with the x axis. The placement of the device is then measured from determination of the slope of lines drawn between the mean centroids of the pad and lead images on opposite sides of the package. It is assumed that the components have been positioned with sufficient accuracy so that soldering pads are visible on all sides. For the cases where the severity of misalignment is such that solder pads are occluded, placement can be determined from two adjacent sides. After the binary image is obtained, processing proceeds as follows: Step I: Objects in the image are roughly classified accord-

ing to centroid coordinates as pads or leads. For each horizontally aligned row, a threshold is chosen for the y coordinate to separate leads from pads; for vertical sides, x coordinate thresholds are used. Examples are shown in Fig. 3. For refer- ence, the pin numbers for each leadtpad pair are indicated and the sides of the device are numbered. For two-sided devices, only sides 1 and 2 are needed. Step 2: Best fit lines are drawn through the centroids of

each row of pads and leads as determined in Step 1. For horizontally oriented rows, the line is chosen such that the sum of the squares of the vertical distances from each centroid point to the line is minimized. For vertical rows, the line which min- imizes the sum of the squares of the horizontal distances is selected. A total of eight lines are generated for the four-sided PLCC; for two-sided SOIC packages, four lines are obtained. Step 3: Pin locations along the best fit line from Step 2 are

estimated. For four-sided devices, two adjacent sides of the component are considered together, as shown in Fig. 4(a). If the best fit line through the horizontal row of leads is denoted as

and that of the vertical edge as

then the intersection (xin, yin) of the lines is given by

The x and y coordinates for the vertical edge are reversed to avoid dealing with large slope values.

For each size of device, both the distance from the intersection to the first lead (I,) and the separation of the leads (SI)

7 y threshold Side I

0 1 2 3

pad images

Side 3 2 Side 4

x threshold - 4 1:: - x threshold I 1

Side 2 y threshold> " "

Fig. 3. Lead/pad images for 18-pin PLCC

-. best fit lines

rr

pad image

estimated first positions (from centroid of large solder pad)

I

are fixed. Therefore, positions of the leads along a best f i t line are easily estimated, as illustrated in Fig. 4(a). This algorithm is robust and gives good estimates of the lead positions even when some objects are missing. A similar procedure is em- ployed to estimate pad positions, using the distance from the intersection to the first pad (p,) and the separation (SI).

For two-sided devices no line intersection can be found. In these cases, the object closest to the first pin position and with significant area is used as the estimated position of the first pin. The estimated positions for other pin numbers then follow at multiples of SI along the line, as shown in Fig. 4(b). (This relies on the location of the first pad although, in our tests, this did not pose a problem. It would, however, be desirable to use a more robust method for future implementations.)

CAPSON AND TSANG. VISUAL MEASUREMENT OF DEVICE PLACEMENT 4 7

F & % d e d from group) - -group together

positions =group centroids

Fig. 5 . Object grouping close to estimated pin position.

Step 4: At each estimated pin location as determined in Step 3 , a more accurate centroid is calculated for each pad and lead area. This step eliminates small areas or combines areas that have been broken up in the image which can occur as a result of the automatic thresholding algorithm, lighting effects, and the resolution of the camera. An example is shown in Fig. 5 .

Each lead normally results in only one object in the image. However, in the event where a single lead gives rise to two or more objects. the centroid is taken as that of the group and is calculated from

b

& P A P

~- mean centroid of pads

(xmp,Ymp)

best fit lines -a@-++- -==z

mean centroid of leads

(Xml SYml) . - centroids of lead/pad regions.

Fig. 6. Mean centroids.

mean centroid of pads (xpm,Ypm)

B

Ya

mean centroid of leads (xlm,ylm) 1 Slope ml

Fig. 7. Distance between mean centroids of pads and leads.

side of the device in the rotated coordinate system is

p = l

p=l

where ( x L p , yep) is the centroid of object p , A , is the area of the object, and b is the number of objects in the group. Objects beyond a specified radius from the estimated pin location are excluded, as shown in Fig. 5. A similar procedure is applied to the solder pad images to obtain accurate pad centroids.

Step 5: Leads and pads on each side corresponding to the same pin number are then paired. If no centroid is found for either the pad or the lead (i.e., no object is close to that estimated position), the pair is discarded. This ensures that only objects which are most likely to be leads and pads are used for further processing. Each pin position is now described by three items: the centroid of the lead, the centroid of the pad, and the pin number of the pair.

Step 6: New lines are then fit through the new centroids of the leads and pads remaining from Step 5 (again, lines are fit to minimize the sum of the squares of vertical or horizontal distance from the centroid points to the line). A mean centroid is calculated for each row of pads and leads on each side of the device. The x coordinate is the mean of the x coordinates of each centroid; the y coordinate is similarly obtained as shown in the example of Fig. 6. The coordinate system is then rotated by an angle I9 so that the best fit line through the pads is aligned with the x axis as shown in Fig. 7. The

where (XI,, ylm) is the mean centroid of the leads and (x,,, y p m ) is the mean centroid of the pads. The angle 8 is obtained from the slope (m,) of the best fit line through the pads

19 = tan-' m p . (7)

The angle a of the best fit line through the leads (of slope ml) with respect to the line through the pads is then

(Y = tan-' ml - tan-' mp. ( 8)

To ensure that mean centroids always refer to the same pin position, a mean pin number is determined from

(9) l r n, = - E n p

where r is the total number of pairs and np is the pin number of the pth pair. Note that n, may not be an integer.

Each side of the device is now described by four parameters: x u , y a , n,, and a. Parameters for all sides are computed using similar procedures and are denoted: ( x , ~ , Y O ] , nmi, ai), ( ~ ~ 2 , yU2, nm2, a d , ( ~ ~ 3 , ~ ~ 3 , nm33 a3), and ( ~ ~ 4 , ya4 , nm4, ( ~ 4 ) for the four sides as labeled in Fig. 3. For two- sided components, only two sets of parameters are required:

p=l

(xu13 YO', nml, &I) and (xa2, ~ a 2 , nm2, ~ Y Z ) .

C . Calculation of Orientation The orientation of a component is then determined from

two opposite sides as shown in the example of Fig. 8. The length of the line (/I) joining the two mean lead centroids is

horizontal (x,) and vertical ( y o ) distances between the mean centroid of the pad and the mean centroid of the lead on each [ I = d2 +(nml - n , , ~ ) ~ s ; (10)

4 8 IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 6, NO. I . FEBRUARY 1990

solder pad number 0

component body

mean centroid of leads

Fig. 8. Calculation of orientation. (a) Length and orientation of 1 1 . (b) Angle of /I for correct alignment. (c) Actual angle of I , .

where d is the minimum distance between two leads on opposite sides and sI is the separation between adjacent leads. If the component is perfectly aligned with the solder pads, as illustrated in Fig. 8(b), line 1 1 will make an angle OP with the horizontal

The actual angle Or that line 1 1 makes with the horizontal is (Fig. 8(c)):

] (12) = cos-i [ ( n m l - n m 2 ) ~ 1 + ( xu1 - x u 2 )

11

Taking the center of the solder pads as the origin, the orientation (0) of the component with respect to the solder pads is then

@ = 8, - 8,. (13)

For example, consider the case when n,l equals nm2. This situation occurs when no lead or pad object on either side is missing. If the component is perfectly aligned with the pads, the line joining the two mean lead centroids will make an angle of a /2 with the horizontal. Since nml equals nm2, the angle 8, equals

x u 1 + X U 2 8, =coscl

and the orientation of the component would be

a @ = - - e (15) 2 ,’

For four-sided packages, a second estimate of @ can be obtained similarly by using the corresponding parameters nm3, nm4, ya3, and ya4 from the vertical sides. The two values of

origin (ceoter of - solder pads)

center of component -

fX.W

mean centroid of pads

i- mean centroid of leads

component body

I--. I

Fig. 9. Calculation of center of component

D. Calculation of Position Once @ is obtained, the location of the center ( X , Y ) of

the component with respect to the center of the solder pads is calculated. For a four-sided component, the x coordinate of the center is obtained from one side (Fig. 9) using

d . cos@> + - 2 sin@ + xul ( 14)

where nl is the total number of pins on the corresponding side of the package. From the opposite side, the x coordinate can be found from

d . SI( 1 - cos@) - -- sin@ + x U 2 . (17)

2

The two estimates for X are then averaged. The y coordinate is computed similarly from the remaining two sides using

d 2 s/(l -cos@) + -s in@ +yu3 (18)

d . sl ( l - cos@)- - s i n 0 + y r r 4 . 119) 2

The results are averaged to give a final value for Y. For devices with only two rows of leads horizontally aligned

in the image, (16) and (17) are applied to compute an average x coordinate of the position of the center of the device. The y coordinate is obtained from the average of (refer to Fig. 10)

111. EXPERIMENTAL RESULTS The placement inspection system was tested with 500

surface-mount components distributed among 50 commercially manufactured, multilayered printed circuit boards. The test samples included 100 of each of the following devices:

a) 84-pin “J” leaded PLCC, b) 48-pin “J” leaded PLCC,

0 are then averaged. c) 18-pin “J” leaded PLCC,

CAPSON AND TSANG: VISUAL MEASUREMENT OF DEVICE PLACEMENT 4 9

center of

( X . W

center of solder pads

mean centroid of pads on side 1

mean centroid of leads , on side 1

mean centroid of pads on side 2 I ' mean centroid of leads on side 2

Fig. 10. Calculation of Y coordinate for two-sided device.

TABLE I PHYSICAL DIMENSIONS OF TEST SAMPLES

Width Length Lead Width Device ("1 ("1 (")

&$-pin PLCC 30.2 30.2 0.76 68-pin PLCC 25.2 25.2 0.76 18-pin PLCC 7.9 13.1 0.76 20-pin SOIC 8.8 12.5 0.64 16-pin SOIC 6.7 11.0 0.64

d) 20-pin gull-wing leaded SOIC, e) 16-pin gull-wing leaded SOIC.

Most of the packages were black in color with the exception of the 16-pin SOIC's which were a mixture of light blue and yellow. The 84-pin and 68-pin PLCC's have an equal number of leads on four sides while the 18-pin PLCC has two sides with five pins and two with four pins. The SOIC's are two-sided packages with the pins divided evenly. The physical dimensions of these devices are shown in Table I.

Positions of the samples were measured manually for com- parison with those obtained by the inspection system. An in- teractive package was developed to allow accurate location of the centroids of leads and pads in a closeup view to obtain manual measurements of position and orientation.

Distributions of the errors e,, e y , and ep (the difference between the system measurement and the manual measurement of horizontal, vertical, and angular displacement, respectively) for each device in the test samples are shown in Figs. 11, 12, and 13, respectively. For each of these figures, part (a) shows results for the PLCC components, while part (b) represent those of the SOIC packages. The corresponding means and standard deviations of the distributions are given in Table 11.

The accuracy of the system can be seen to be consistent (with means near zero) for the measurement of displacement in x and y directions for both the PLCC and the SOIC components of the various sizes used in the testing. If the 3a limits are considered, the system provides an accuracy ranging from approximately 0.09 to 0.25 mm for x displacement, 0.08 to 0.13 mm for y displacement, and 0.23" to 1.74" for angu-

0

30

n

c 20 Y

8

10

0 - 0 6 -0 4 -0.2 0 0.2 0.4 0.6

Error In X (mm)

-+ 68 pln PLCC - 84 pin PLCC + 16 Pin PLCC

Y Y

e h U 30 I

; i , i -OB - 0 4 -02 0 0 2 0 4 0 6 1 0.6

Error in X (mm) - 20 pln SOlC + 16 pin SOlC

(b) Fig. 11. Horizontal displacement error. (a) PLCC. (b) SOIC

50

40 k I

-0.3 -0.2 -0 1 0 0.1 0.2 0.3 Error in Y (mm)

- 20 pln SOIC + 16 pln SOlC

(b)

Fig. 12. Vertical displacement error. (a) PLCC. (b) SOIC

lar displacement with the best results for the larger packages. The most notable difference is in the error distribution for angular displacement from which can be seen the decreased performance for the smaller 18-pin PLCC (Fig. 13(a)) and the SOIC devices (Fig. 13(b)).

As a test of repeatability, 100 measurements of horizontal position, vertical position, and angular orientation for a single 84-pin PLCC were obtained by the automatic system.

50 IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION. VOL 6. i%O I , F E B R U A R Y 1990

80

70 -

60 -

50-

4 0 -

30-

2 0 -

10 -

0

F

e

-3 -2 -1 0 1 2

TABLE I1 MEAN AND STANDARD DEVIATION OF ERROR DISTRIBUTION

~~

~~

3

Device

e, (degree?)

mean S D

84-pin PLCC 0.0068 0.041 - 0.0013 0.027 0.032 0.076 68-pin PLCC 0.0010 0.031 0.0043 0.032 - 0.032 0.082 18-pin PLCC 0.0041 0.082 0.0081 0.044 - 0.075 0.58 20-pin SOIC - 0.0236 0.077 0.0013 0.039 - 0.55 0.40 16-pin SOIC - 0.0009 0.079 0.001 I 0.027 0.229 0.48

F

e

9 e 60-

n 4 0 -

U

r

80 -

C Y

20

0 -3 -2 -1 0 1 2 3

Error In Orlentation (degrees)

TABLE I11 MEAN AND STANDARD DEVIATION OF REPEATABILITY TEST FOR

&-PIN PLCC ~~

______________~___~ _ _ ~

Horizontal Vertical Angle ("1 (degrees)

Mean 0.047 - 0.092 0.093 S.D. 0.003 0.005 0.009 Manual measurement 0.036 - 0.103 0.023

The circuit board was repositioned within the field of view for each measurement. The mean and standard deviations of the distributions together with the manual measurements are summarized in Table 111. Considering the 3a limits for these distributions, the system consistently reports x and y displacements within approximately 0.09 and 0.0 15 mm, respectively, of the manual measurements. The angular repeatability is in the range of 0.03" of the manual measurement.

It is generally accepted that a "J" lead should not be displaced by more than 50% of the lead width from a pad in any direction to ensure reliable solder joints (due to the longer

(nl-l)s$2 ,-end solder pad r-" i Y

origin (center of - solder pads)

end lead

Component body

position when ,- perfectly aligned

center of component

origin (center of

i i

I

(b) Fig. 14. Displacement of end lead. (a) 4-sided device. (b) ?-sided device

leads, the corresponding figure for gull-wing type leads is 25%). The tolerances that satisfy these requirements for the different devices in our test samples from the lead widths listed in Table I are 0.38 mm for the PLCC devices and 0.16 mm for the SOIC components.

The center of the component ( X , Y ) and orientation ( p ) can be used to determine the maximum displacement of a lead from a soldering pad. This will always be a lead at the end of a row. Referring to Fig. 14(a), the maximum horizontal displacement will be

and the maximum vertical displacement is given by

CAPSON AND TSANG VISUAL MEASUREMENT OF DEVICE PLACEMENT 5 1

TABLE IV CLASSIFICATION RESULTS BASED ON END LEAD DISPLACEMENT

Device Acceptable Reject False Alarm Miss

84-pin PLCC 94 6 0 0 68-pin PLCC 100 0 0 0 18-pin PLCC 100 0 4 0 20-pin SOIC 90 10 2 10 16-pin SOIC 94 6 6 1

For two-sided devices, horizontally aligned in the image, the horizontal displacement is obtained from (22) while the maximum vertical displacement is given by (refer to Fig. 14(b))

The performance of the system based on detection of maximum displacement of leads from solder pads has been measured and the results are given in Table IV. For the 84-pin and 68-pin PLCC devices, the system was able to correctly classify placement accuracy for all of our test samples. For example, of the 100 84-pin components, 94 were actually positioned properly (the end lead was not displaced by more than 50% of the lead width) and 6 were placed incorrectly. No “false alarms” (correctly placed components reported as faulty) or “misses” (incorrectly placed components reported as acceptable) were generated. For the 18-pin PLCC packages, 4 false alarms were determined.

Classification of SOIC placement was not as reliable. For the 20-pin device, 10 misses and 2 false alarms were reported; for the 16-pin SOIC, the system produced 6 false alarms and 1 miss.

IV. DISCUSSION The experimental system has demonstrated an economical

configuration for accurate measurement of the placement of J-leaded and gull-winged surface-mount devices on printed circuit boards. Since the inspection is achieved from a single camera view of both soldering pads and leads, the method does not rely on the precision of either the component loading device or the mechanism ( X Y table) used to position the circuit board, both of which can exhibit accumulating errors. The system is most effective for the larger (84-pin and 68- pin) PLCC packages; however, decreased performance for the SOIC components is not unexpected due to the longer leads and larger soldering pads. For these devices, more of the soldering pads are covered by the leads, even small displacements have a relatively large effect in determining the location of the pad centroids.

The average error over all devices tested were

0.0383 mm in the x direction 0.0233 mm in the y direction 0.306’ for angular displacement.

More precision is achieved vertically due to the greater resolution of our sensor in this direction. For classification of acceptable/unacceptable placement of components based on the maximum displacement of the end leads, the system accuracy

ranged from 100% for the larger PLCC’s to 88% for the 20- pin SOIC.

The infrared illumination was found to be an effective technique for producing high-contrast images of the leads and pads. Although all of the results reported here are for soldered devices, we anticipate that the system would be suitable for presoldering inspection since the solder paste used to hold components on the board is sufficiently different in color from the board materials to obtain good images.

If a component is misplaced so as to cover solder pads, it may not be possible to determine placement and orientation as previously described. In these cases, the system can simply reject the component or, if it is desirable to feed back an error signal to the loading device, the calculations of placement can be achieved for the PLCC using two adjacent sides rather than opposite sides of the package. The appropriate geometric equations have been developed for this purpose but are not included in the results shown here.

The performance of the system could be enhanced with the use of a faster computer and numeric coprocessor which are now readily available. With the current 6-MHz, 80286-based IBM PC-AT processor, measurement speeds are typically 2-4 s per component for our test samples. Moreover, the image analysis tasks could be supplemented with high-speed hard- ware designed for fast image preprocessing. For an automated assembly line, multiple sensors could be provided to minimize the movement of the board within the inspection station and to maximize the resolution available within the field of view for each size of component. Future work might involve im- provements along these lines as well as the addition of lead and pad count verification, the detection of bent leads, and of course adaptation to and performance evaluation in a produc- tion environment. Also, if the solder pads were considered to be “targets,” circuit board warpage may be measured using photogrammetric techniques.

REFERENCES H. Test, L. Woodson, and C. Hutchins, “Key issues of surface mounted ICs,” in Tech. Papers on Surface Mount Technology 1984-1986 (Texas Instruments, 1986), pp. 1-7. C. C. Hefner, “Electrical performance enhancements due to surface mount packaging,” in Tech. Papers on Surface Mount Technology 1984-1986 (Texas Instruments, 1986), pp. 33-38. H. H. Manko, Soldering Handbook for Printed Circuits and Sur- face Mounting. R. J . K. Wassink, Soldering in Electronics. Pennington, NJ: Elec- trochemical Publ., 1984. J. Mullen, How to Use Surface Mount Technology. Dallas, TX: Texas Instruments, 1984. Introduction to Surface Mounting (MOS Memory Products Appli- cation Rep). Dallas TX: Texas Instruments, 1984. R. T. Chin and C. A. Harlow, “Automated visual inspection: A sur- vey,” IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-4, no.

J. L. C. Sanz and A. K. Jain, “Machine-vision techniques for inspection of printed wiring boards and thick-film circuits,” J . Opt. Soc. Amer. A , vol. 3, no. 9, pp. 1465-1482, 1986. Y. Hara, H. Doi, K. Karasaki, and T. Ilda, “A system for PCB automated inspection using fluorescent light,” IEEE Trans. Pattern Anal. Machine Intell., vol. 10, no. 1, pp. 69-77, 1988. G. Goto and T. Kondo, “An automatic inspection system for printed wiring board masks,” Pattern Recogn., vol. 12, pp. 443-455, 1980. G. A. W. West, “A system for the automatic visual inspection of bare- printed boards,” IEEE Trans. Syst., Man, Cybern., vol. SMC-14, no. 5, pp. 767-773, 1984.

New York, NY: Van Nostrand Reinhold, 1986.

6, pp. 557-573, 1982.

IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION. VOL 6. NO I . FEBRUARY IYYO

Y. Nakagawa, “Automatic visual inspection of solder joints on printed circuit boards,” Robotic Vision (SME), vol. 336, pp. 121-127, 1982. W. E. McIntosh, “Automating the inspection of printed circuit hoards,’’ Robotics Today, pp. 75-78, 1983. S. L. Bartlett, P. J. Bed, C. L. Cole, R. Jain, D. Mukherjee, and K. D. Skifstad, “Automatic solder joint inspection,” IEEE Trans. Pattern Anal. Machine Infel l . , vol. 10, no. 1, pp. 31-43, 1988. D. W. Capson and S. K. Eng, ”A tiered-colour illumination approach for machine inspection of solder joints,’’ IEEE Trans. Pattern Anal. Machine Intell., vol. 10, no. 3, pp. 387-393, 1988. W. E. Blanz, J . L. C . Sanz, and E. B. Hinkle, “Image analysis meth- ods for solder-ball inspection in integrated circuit manufacturing,” IEEE J . Robotics Automat., vol. 4, no. 2, pp. 129-139, 1988. P. A. Merill, “Automatic visual inspection of solder joints,” M.Eng. thesis, McGill Univ., Montreal, Que., Canada, 1984. S. Okamoto, K. Yoshimura, and T. Nakahara, “Soldering inspection system and method therefor,” U.S. Patent 4 677 473, 1987. A. C. Traub. “Thermal energy as a diagnostic tool,” Sensors J., vol. 2, no. 3, pp. 17-29, 1985. M. P. Seah and C. Lea, “Certainty of measurement using an automated infrared laser inspection instrument for PCB solder joint integrity,” J . Physics E: Sci. Instrum., vol. 18. pp. 676-682, 1985. M. Juha, “X-ray machine vision for circuit board inspection,” in SME Vision ’86 Conf. Proc., p. 3-41-3-55, 1986. M. G. Buffa, “Process control for automated component assembly of surface mounted devices utilizing a machine vision controller,” in SME Vision ’85 Conf. Proc., pp. 5-180-5-200, 1985. N. S. Chang, “SMV-A computer vision program for ioading surface mount components,” Proc. SME, vol. 557, pp. 76-80, 1985. K. Susuki et al., “Method of loading surface mounted devices and an apparatus therefor,” U.S. Patent 4 737 845, Apr. 12, 1988. N. Otsu, “A threshold selection method from gray-level histograms,” IEEE Trans. Syst., Man, Cybern., vol. SMC-9, no. 1, pp. 62-66, 1979.

[26] D. W. Capson, “An improved algorithm for the sequential extractmn of boundaries from a raster scan.“ Commit. Vision. Graoh. Irnaw Process., vol. 28 pp. 109-125, 1984

David W. Capson (S’79-S’82-h.1‘8?~M‘84) I.\ an Assistant Professor in the Department of Electrical and Computer Engineering at McMaster C1nivt.r- sity, Hamilton, Ont.. Canada. and is a member of the Manufacturing Research Corporation of Ontario (MRCO) established in the Centers of Excellence program. His experience includes the development of machine vision algorithms as well as the dejign and implementation of computer vision systems for applications in automated manufacturing. Current research interests include automated systems for in-

spection of printed circuit boards. blood-cell motion tracking, and real-time object tracking. From January to June of 1989. he worked in the machine vision group of the Advanced Algorithms, Architectures and Applications Department at the IBM Almaden Research Center, San Jose. CA.

Dr. Capson is a member of the Society of Manufacturing Engineers ISME, and is a Licensed Professional Engineer in the Province of Ontario.

Randy M. C. Tsang was born in Hong K o n p and obtained his primary and secondary educu- tion there. He received the Bachelor‘s degree in computer engineering from McMaster IJniveruty. Hamilton, Ont.. Canada, in 1986 a n d the Master’\ degree in 1988.

Currently, he is working for Schlumberger Wire- line Services for oil exploration in Egypt and India.

automatic visual measurement of surface-mount device placement

Documents