automatic image processing for welding inspection
TRANSCRIPT
Automatic Image Processing For Welding Inspection
CATALIN GHEORGHE AMZA, DAN NITOI FLORIN, ENCIU GEORGE
Materials Technology and Welding Department
University “Politehnica” of Bucharest
Splaiul Independentei 313, sector 6, Bucuresti
ROMANIA
[email protected] http://www.upb.ro
Abstract: - This paper deals with the automatic detection of defects within various welding. The authors
propose a method based on taking an X-ray image of the welding and then applying various image processing
algorithms for the detection of possible defects. This method has a significant improvement in the detection
success ratio compared to classical detection techniques (by using human operators).
Key-Words: defect detection, industrial inspection of welding, image processing
1 Introduction There are just a few products of industrial processes
so well defined that their quality can be guaranteed
to meet exactly the client specifications or
requirements. However, in most areas of industrial
processing some sort of testing or inspection has to
be performed on intermediate or final products [1],
[2]. When dealing with small and simple batch
products, it is easy and inexpensive to perform
destructive testing on a sample of components, but
in most of the industrial cases, non-destructive
inspection techniques are employed.
Fig. 1 General X-ray image-based industrial
inspection system
Many modern inspection systems as depicted in
Fig.1 are based on processing an image taken from
an inspected product. The image can be obtained by
means of irradiating the product and using a normal
or infrared television camera. Thus, an important
role in this process is played by the image
acquisition sub-system. This sub-system transforms
physical signals taken from the inspected product
into useful visual signals [3]. Image processing
techniques are then employed to enhance and
analyse the resultant image and with the aid of an
expert knowledge database the decision whether the
product can pass the inspection tests is taken. The
latter methods are the most used across the food [3]
and manufacturing industry and they are based on
the use of some kind of irradiation of the product
and deduces from the intensity of the radiation
whether there is a defect present or not. X-rays are
currently used in most of those systems [4] and it
was originally proposed from medical science where
an X-ray is used for detection of foreign matter in
the human body. X-ray images were captured and
transmitted directly to the computer for further
processing and for automatic inspection [5]. The
acquired images need to be pre-processed in order to
be enhanced and possible noise be removed [6].
Then, the image is segmented and analysed. The
current papers presents image processing techniques
for welding. A sample part is presented in Fig. 2.
Fig.2 Welding part used for quality inspection
2 Image pre-processing In the X-ray images taken, the raw information is in
most of the cases presented in a way unsuitable for
Recent Advances in Industrial and Manufacturing Technologies
ISBN: 978-1-61804-186-9 153
the human eye or for further image analysis
techniques (poor contrast, presence of noise, etc.).
Therefore, methods for “improving” or enhancing
the images were developed. The contrast of an
image is given by the distribution of its grey levels
or its pixel values. If this distribution is concentrated
near a certain level, then the contrast is obviously
low. On the other hand, when a wide range of grey
levels are presented in the image, then the contrast is
high. It is the main objective of image pre
processing to improve images to appear suitable for
the human eye. Only after pre-processing, further
automated computer based image analysis
algorithms can be employed, since all are based on
the knowledge gathered through human visual
senses.
Transmission over cables of video signals is often
affected by the presence of electromagnetic fields,
bad shielding of cables, etc. Consequently, noise can
corrupt the resultant images. The noise pattern
consists of strong, spike like components that can
affect randomly the grey-value of one or more
pixels, called impulse or salt-and-pepper noise.
Methods used for noise removal are usually based
upon spatial-domain techniques. In the spatial
domain, an image-processing function can
expressed as follows:
)],([(),( yxfTransfyxg =
where f(x,y) is the input image, g(x,y)
processed image, and Transf is a spatial operator on
f defined over some neighbourhood of
Usually, a rectangular mask or window centred in
(x,y) is moved from pixel to pixel . The operator is
then applied to compute the processed image
The image can be smoothed by averaging the grey
level values of the pixels in a predefined
neighbourhood. This spatial-domain technique is
called neighbourhood averaging and can be
successfully applied for the removal of impulse
noise:
0..Nx,y,qpft
yxgoodNeighbourhgp
== ∑∈
,),(1
),(),(
where t is the number of points in the
neighbourhood. The main disadvantage of this
technique is that it blurs edges and other sharp
details. Since defects are considered areas of sharp
details, a median filtering was used to remove the
impulse noise. The method consists in replacing the
grey-value of each pixel by the median of the grey
levels in a neighbourhood (or pre-specified window
the human eye or for further image analysis
rast, presence of noise, etc.).
Therefore, methods for “improving” or enhancing
the images were developed. The contrast of an
stribution of its grey levels
or its pixel values. If this distribution is concentrated
near a certain level, then the contrast is obviously
low. On the other hand, when a wide range of grey
levels are presented in the image, then the contrast is
the main objective of image pre-
processing to improve images to appear suitable for
processing, further
automated computer based image analysis
algorithms can be employed, since all are based on
gh human visual
smission over cables of video signals is often
affected by the presence of electromagnetic fields,
bad shielding of cables, etc. Consequently, noise can
corrupt the resultant images. The noise pattern
like components that can
value of one or more
pepper noise.
Methods used for noise removal are usually based
domain techniques. In the spatial
processing function can be
(1)
g(x,y) is the
is a spatial operator on
defined over some neighbourhood of (x,y).
Usually, a rectangular mask or window centred in
) is moved from pixel to pixel . The operator is
then applied to compute the processed image g.
The image can be smoothed by averaging the grey-
level values of the pixels in a predefined
domain technique is
averaging and can be
successfully applied for the removal of impulse
(2)
is the number of points in the
neighbourhood. The main disadvantage of this
technique is that it blurs edges and other sharp
are considered areas of sharp
details, a median filtering was used to remove the
impulse noise. The method consists in replacing the
value of each pixel by the median of the grey
specified window
or filtering mask) of that pixel, instead of by the
average. In this way, points or pixels with very
distinct intensities will be removed and replaced
with pixels more like their neighbours. A 3 by 3
pixels window was employed and the median filter
was applied for the acquired X-
passes. The results can be seen in Fig. 3
Fig. 3 Noise removal by using a 3 by 3 pixels
median filter (2 passes) in weldings X
(left - original image corrupted by impulse noise;
right - result after median
Due to the use of an image intensifier
acquisition [6], intensified images of a
object are generally brighter in the centre than in the
periphery due to an unequal brightness gain in
different regions of the field of view. This effect is
also called vignetting (see Fig. 4
performance of the X-ray tube and the fluorescent
screen of the image intensifier suffer decay over
time. Further analysis of the images may suffer over
this effect and therefore, calibration images are
taken at the start of each inspectio
Fig.4 Vignetting effect for X-ray image intensifiers
An example of image calibration
Fig. 5. By simply substracting the calibration image
from the input image, the vignetting effect can be
removed and the decay of X
system’s performance over time is annulled. There
sk) of that pixel, instead of by the
average. In this way, points or pixels with very
distinct intensities will be removed and replaced
with pixels more like their neighbours. A 3 by 3
pixels window was employed and the median filter
-ray images in two
esults can be seen in Fig. 3.
moval by using a 3 by 3 pixels
) in weldings X-ray images
e corrupted by impulse noise;
result after median filtering)
Due to the use of an image intensifier during
, intensified images of an uniform
object are generally brighter in the centre than in the
periphery due to an unequal brightness gain in
different regions of the field of view. This effect is
ig. 4). Moreover, the
ray tube and the fluorescent
screen of the image intensifier suffer decay over
time. Further analysis of the images may suffer over
this effect and therefore, calibration images are
inspection process.
ray image intensifiers
An example of image calibration is illustrated in
By simply substracting the calibration image
from the input image, the vignetting effect can be
removed and the decay of X-ray acquisition
system’s performance over time is annulled. There
Recent Advances in Industrial and Manufacturing Technologies
ISBN: 978-1-61804-186-9 154
is no need for the registration of the two images and
where needed, the result was normalized by dividing
the average pixel intensity value of the original
image [6]. Also the removal of the bias
the actual image, process called debiasing, is
achieved in this way.
The results are shown in Fig. 6.
Fig.5 Calibration images for low and high energy
views
Simple arithmetic techniques from the spatial
domain are employed for contrast enhan
the resultant filtered X-ray images. By multiplying
an image by itself (square the image) the intensity
values of importance can be enhanced (contrast
enhancement [7]). Normalisation can sequentially
be performed by dividing the average pixel int
value of the original image in order to achieve a
high contrast image. Various pre-processed images
of X-ray welding are presented in Fig. 6.
Fig. 6 Various pre-processed X-ray images
The next step of the image processing system is the
image segmentation process.
is no need for the registration of the two images and
where needed, the result was normalized by dividing
the average pixel intensity value of the original
. Also the removal of the bias frame from
the actual image, process called debiasing, is
Calibration images for low and high energy
Simple arithmetic techniques from the spatial-
domain are employed for contrast enhancement of
ray images. By multiplying
an image by itself (square the image) the intensity
values of importance can be enhanced (contrast
). Normalisation can sequentially
be performed by dividing the average pixel intensity
value of the original image in order to achieve a
processed images
.
ray images
The next step of the image processing system is the
3 Image segmentation The segmentation process is
important image processing techniques
an image-based inspection system. It is the process
of clustering or dividing the image into areas of
context-specific meaning. A segmentation algorithm
for a welding X-ray image needs to separate
(such as cracks, gas inclusions, poros
background (the product itself)
separating not only entire objects from the
background, but also separating only parts of objects
from the background as this is also considered a
successful technique [9].
A multi-thresholding of an X
provide a useful result for further image analysis
techniques due to high sharpness of the defects
illustrate this, an Otsu-based
algorithm was implemented [10], [11
result for segmentation intro 3 classes (the number
of classes was determined experimentally) is
depicted in Fig. 7.
Fig.7 Results of Otsu’s thresholding method
a) Original X-ray image; b) Segmented image into 3
classes
4 Post processing and high
detection Once the segmentation process is over, the resulting
segments or objects need to be analysed and a
decision has to be taken whether they are
not. This is a process of classifying the objects as
The segmentation process is one of the most
techniques employed in
based inspection system. It is the process
of clustering or dividing the image into areas of
segmentation algorithm
ray image needs to separate defects
cracks, gas inclusions, porosities) from the
itself) [8]. One aims in
separating not only entire objects from the
background, but also separating only parts of objects
is also considered a
X-ray image would
result for further image analysis
due to high sharpness of the defects. To
multi-thresholding
[10], [11], [12]. The
for segmentation intro 3 classes (the number
of classes was determined experimentally) is
a)
b) Results of Otsu’s thresholding method
ray image; b) Segmented image into 3
Post processing and high-level
Once the segmentation process is over, the resulting
segments or objects need to be analysed and a
decision has to be taken whether they are defects or
not. This is a process of classifying the objects as
Recent Advances in Industrial and Manufacturing Technologies
ISBN: 978-1-61804-186-9 155
being defects or not and it is considered a pattern
recognition procedure.
Practical image recognition systems generally
contain several stages in addition to the recognition
itself. A general pattern recognition system can
comprise a feature extraction unit, a classifier unit
and a context processor (see Fig. 8 [13]). From the
input objects or patterns (segmented objects), the
feature unit extracts information in order to facilitate
recognition. The classifier unit identifies which
pattern the object belongs to. The optional context
processor increases recognition accuracy by
providing relevant information regarding the
environment surrounding the object.
Fig. 8 Components of a pattern recognition
system [13]
In the present case, the analysis of the segmented
image can be regarded as a PR problem. Segmented
images contain the patterns that need to be classified
or recognized. Since the segmentation was
performed using 3 levels, the resulting partitioned
image has objects characterized by 3 grey levels.
The methodology followed is presented in Fig. 9.
Firstly, for each segmentation level, an object is
extracted. Then, for that object features are
extracted. Finally, based on the features extracted
from the object, this object is classified either a as a
defect (i.e. crack) or normal product. Another object
is then extracted and the procedure is repeated for
all objects and for all segmentation classes.
However, at any moment, if an object is found to be
a defect, the algorithm stops, and the product is
rejected.
Fig. 9 Summary of the defect detection system
The object extraction process extracts the objects
from a segmentation level one by one. For each
segmentation level, a backtracking algorithm was
designed and implemented for the object extraction.
It consists of a function that requires a starting point
(pixel) of the current object to extract. Then, all
pixels adjacent to the starting pixel are tested for
similarity. If a pixel is found to be the same
grey-level as the object, then it is marked
respectively. Otherwise the process is stopped. The
algorithm is called backtracking due to the fact that
the function is called recursively within itself (see
below). The algorithm searches for neighbour pixels
from right to left for instance and it continues to do
so until a dead end is reached; then the function
“backtracks” its way to the starting point and the
process continues for considering pixels from left to
right and so on.
When the function has marked all the pixels of an
object, features are extracted from it and further
high-level classification is performed. As explained
before, if the object analysed is found to be a defect,
then the entire inspection process is stopped and the
product is rejected. Otherwise, the object is simply
erased for the analysed class and another one is
extracted using the same backtracking technique.
Backtracking algorithm for object extraction
function Object_extraction(starting point(x,y)) begin
if starting_point(x,y) (does not belong to the object segmentation level) and (it was considered before) then STOP else //Pixel belongs to the object and it was not
//considered before begin mark (x,y) as (already being considered);
//start and check the neighbouring pixels Object_extraction(x+1,y); Object_extraction(x-1,y); Object_extraction(x,y-1); Object_extraction(x,y+1);
end; end;
The starting point of an object is obtained by
scanning the segmented image line by line until the
first pixel within the current segmentation level is
found. That pixel is considered as being the starting
point of a new and not considered object within the
analysed class.
4.1 Feature extraction Classical mathematical methods were employed in
the present case for computing the geometrical or
logical features. Among these features, the
following were computed:
• The physical area size of the current object
(measured in pixels) AREA_SIZE, variable
Recent Advances in Industrial and Manufacturing Technologies
ISBN: 978-1-61804-186-9 156
that can be regarded as the total number of
pixels in the current object;
• The perimeter of the current area
PERIMETER: which is the number of
pixels on the edge that surrounds
• The average grey-level intensity of the
object BRIGHTNESS, computed as
• The difference in intensity between the
object and a rectangular area that surrounds
it - DIFFERENCE; a surrounding rectangle
was defined for computing the difference
measure as can be seen on Fig. 10 [6]
Fig. 10 Examples of surrounding rectangle
of two extracted objects [6]
Other features were extracted wherever there was a
need for more precise characterisation of the
objects. Once the features are extracted, they need to
be analysed and compared with stored or known
patterns in order to take a final decision. Thus, high
level classification is required, and the process will
be presented in the following paragraphs.
The features were extracted from 50 segmented
images; this rendered about 785 objects of which
135 were defects regions and 650 were normal
product regions. From all 785 objects, input vectors
consisting of features were extracted. These input
vectors were divided into training and testing data,
vectors needed for training and testing the proposed
architecture image processing methods.
Once the features are extracted from an object, the
simplest way is to compare them with the
information already stocked within a know
database (or a simple look-up table). The knowledge
base is usually designed and collected using expert
human observations. Sample images are analysed by
human experts and the information is saved in a
useful format in an expert database. The
methodology is presented in Fig. 11. Four
were used for each pattern in this classical approach
to PR:
• BRIGHTNESS – average X-ray intensity of
the region;
• DIFFERENCE – difference between
BRIGHTNESS and the average intensity
within a surrounding rectangular area;
that can be regarded as the total number of
The perimeter of the current area
PERIMETER: which is the number of
s the object;
level intensity of the
object BRIGHTNESS, computed as in [6];
The difference in intensity between the
object and a rectangular area that surrounds
DIFFERENCE; a surrounding rectangle
ned for computing the difference
re as can be seen on Fig. 10 [6].
surrounding rectangle
Other features were extracted wherever there was a
need for more precise characterisation of the
Once the features are extracted, they need to
be analysed and compared with stored or known
patterns in order to take a final decision. Thus, high-
level classification is required, and the process will
be presented in the following paragraphs.
s were extracted from 50 segmented
images; this rendered about 785 objects of which
regions and 650 were normal
regions. From all 785 objects, input vectors
consisting of features were extracted. These input
to training and testing data,
sting the proposed
Once the features are extracted from an object, the
simplest way is to compare them with the
information already stocked within a knowledge
up table). The knowledge
using expert
observations. Sample images are analysed by
human experts and the information is saved in a
useful format in an expert database. The
Four features
were used for each pattern in this classical approach
ray intensity of
difference between
BRIGHTNESS and the average intensity
rectangular area;
• AREA_SIZE – the physical area of the
region;
• PERIMETER – the physical perimeter of
the region;
Fig. 11 Classical approach to PR using
a look-up table
The implemented look-up table consists of already
stored patterns defined using hard
The main criteria for a defect, extracted from the
expert knowledge database that was gathered
through experiments, is defined as summarised in
Table 1.
Table 1 Defects definition
Feature Hard if-then
AREA_SIZE Between 150 and 52
PERIMETER Between 0 and 5000 pixels
BRIGHTNESS Between 180 and 250 (grey level values)
DIFFERENCE Between 230 and 255 (grey level values)
Objects with any other characteristics or features are
considered as being normal product
The performance of such a classical look
approach based on human experience database was
on average at 86.3% for the tested 50 images
example of detected defect for a welding
in Fig. 7 is depicted in Fig. 12.
Fig. 12 Example of an analysed image with three
detected defects
the physical area of the
the physical perimeter of
Classical approach to PR using
up table
up table consists of already
stored patterns defined using hard if-then decisions.
, extracted from the
expert knowledge database that was gathered
fined as summarised in
definition
then condition
150 and 5200 pixels
00 pixels
Between 180 and 250 (grey level values)
Between 230 and 255 (grey level values)
Objects with any other characteristics or features are
product.
classical look-up table
approach based on human experience database was
for the tested 50 images. An
ed defect for a welding presented
Example of an analysed image with three
detected defects
Recent Advances in Industrial and Manufacturing Technologies
ISBN: 978-1-61804-186-9 157
5 Conclusion A human based inspection system has the property
that the final decision is taken by a human operator.
In most of the cases, the operator views the X-ray
image of the inspected product on a monitor. Based
on that image and his/her knowledge and experience
the operator has the ability to reject or accept the
inspected product.
However, humans are known to be subjective when
making a classification. Also different levels of
training and experience can render different results
for different operators. Furthermore, human factors
such as physical stimuli and the physical and mental
state of the operator at the time of the inspection
influence the outcome of the process [14], [15]. The
surrounding environment can also make the
operator-based inspection process a variable one
(for instance, in different light conditions the
decision can be changed dramatically due to change
perceived on the monitor’s screen) [16], [17].
This paper presented various image processing
techniques for the automated quality inspection of
weldings. In such a system, human intervention is
none or minimum. Main advantage is that the
system is not influenced by the human factors
presented above. The algorithms presented proved
the viability of such approach in an automated
welding quality inspection system based on the
acquisition of an X-ray image and its subsequent
automated analysis.
References:
[1] Kehoe, A., The detection and evaluation of
defects in industrial images, PhD Thesis,
University of Surrey, 1990
[2] Kehoe, A., Parker, G.A., An intelligent
knowledge based approach for the automated
radiographic inspection of castings, NDT&E
International, vol. 25, no.1, pp.23-36, 1992
[3] Graves, M., Approaches to foreign body
detection in foods, Trends in Food Science and
Technology, vol.9, January 1998
[4] Connolly, C., X-ray systems for security and
industrial inspection, Sensor Review, Vol. 28
Iss: 3, pp.194 – 198, 2008
[5] Amza, C.G., Amza, G., Cicic, D.T., Popescu
D., Modern industrial foreign body detection
systems, Proceedings of the 12th International
Conference on Technology and Quality for
Sustained Development, ISSN 1844-9158, pag.
11-20, 2012
[6] Amza, C., G., Intelligent X-ray Imaging
Inspection System for the Food Industry, PhD
Thesis, De Montfort University, Leicester,
United Kingdom, 2002
[7] Kaur M., Kaur J., Survey of contrast
enhancement techniques based on histogram
equalization, International Journal of advanced
Computer Science and Applications (IJACSA),
vol2, no.7, pp.137-141, 2011
[8] Amza, C.G., Tasca, G., Segmentation of
Industrial X-ray images, Proceedings of the 4th
WSEAS International Conference on
Computational Intelligence CI’10, Bucharest,
Romania, 2010, pp.54-59
[9] Amza, C.G., Amza, G., Popescu D., Image
segmentation for industrial quality inspection,
rev. Fiability and Durability, Supplement
no.1/2012, Ed. “Academica Brancusi”, Targu
Jiu, pp.126-132, 2012
[10] Otsu, N., A threshold selection method from
grey-level histograms, IEEE Trans. On systems,
man and Cybernetics, vol. SMC-9, no.1, pp.62-
66, 1979
[11] Ch.Hima Bindu, An Improved Medical
Image Algorithm Using Otsu Method,
International Journal of Recent Trends in
Engineering, Vol 2, No. 3, November 2009
[12] Leung, C.K., Lam, F.K., Maximum
segmented image information thresholding,
Graphical Models and Image Processing,
vol.60, no.1, pp.57-76, 1998
[13] Chen, C., Statistical Pattern Recognition,
Hayden, Washington, 1973
[14] Chapanis, A., Human factors in systems
engineering. New York: John Wiley & Sons;
1996
[15] Reason, J., Human Error. Cambridge,
Cambridge University Press, 1992
[16] Kaye, R., Crowley, J., Medical-device use
safety: Incorporating human factors engineering
into risk management, U.S. Food and Drug
Administration, July, 2000
[17] Stanciu, M.I.; Nicolescu, A.; Enciu, G.;
Ghinea, M.: Intelligent Vision System for
Grasping Error Compensation of a Wafer
Manipulation Robot" The 19th International
Daaam Symposium " Intelligent Manufacturing
& Automation: Focus On Focus On Next
Generation Of Intelligent Systems And
Solutions", 22-25TH October 2008, Trnava,
Slovakia, ISSN 1726-9679, pg. 1297 - 1298
Recent Advances in Industrial and Manufacturing Technologies
ISBN: 978-1-61804-186-9 158