Segmentation of Fingerprint Images Based on Bi-Level Processing Using Fuzzy Rules

Hasan Fleyeh Erfan Davami Diala Jomaa Computer Engineering Department Computer Engineering Department Computer Engineering Department

Dalarna University Dalarna University Dalarna University Borlänge, Sweden Borlänge Sweden Borlänge Sweden

hfl@du.se V09erfda@du.se djo@du.se Abstract - This paper presents a new approach to segment low quality fingerprint images which are collected by low quality fingerprint readers. Images collected using such readers are easy to collect but difficult to segment. The proposed approach is based on combining global and local processing to achieve segmentation of fingerprint images. On the global level, the fingerprint is located and extracted from the rest of the image by using a global thresholding followed by dilation and edge detection of the largest object in the image. On the local level, fingerprint’s foreground and its border image are treated using different fuzzy rules which the two images are segmented. These rules are based on the mean and variance of the block under consideration. The approach is implemented in three stages; pre-processing, segmentation, and post-processing. Segmentation of 100 images was performed and compared with manual examinations by human experts. The experiments showed that 96% of images under test are correctly segmented. The results from the quality of segmentation test revealed that the average error in block segmentation was 2.84% and the false positive and false negatives were approximately 1.4%. This indicates the high robustness of the proposed approach.

I. INTRODUCTION

A captured fingerprint image usually consists of two components; the foreground and the background. The foreground is the area of scanner surface which is in contact with the finger surface. It includes the necessary information needed for fingerprint recognition, while the background is the noisy area which is located at the borders of the image. Fingerprint segmentation is the process by which the foreground is separated from the image background. The result of fingerprint segmentation is a fingerprint image in which the background is removed [1; 2]. Fingerprint segmentation is an important step in automatic fingerprint recognition systems because it improves the fingerprint images so that features can be extracted from these images by the automatic fingerprint identification systems. Features such as minutiae and singular points are especially important for the reliable identification of fingerprints. When fingerprint images include a noisy background, feature extraction algorithms extract many false features. Hence, developing good fingerprint segmentation algorithm helps to discard the background, and thus reduces the number of false features.

Most of the segmentation algorithms aim to use one level of features to achieve segmentation [1; 3; 4; 5; 6; 7; 8], which is achieved by two approaches: block-wise based or pixel-wise based [9]. In the block-wise approach, the fingerprint image is divided into blocks and each block is classified into foreground or background based on features calculated for the block. While in the pixel-wise method, segmentation is achieved on the pixel level. In order to achieve better discrimination between the foreground and the background of the fingerprint and to reduce the misclassification error, this paper presents a new algorithm for the segmentation of fingerprint images. Segmentation is achieved by using a bi-level combination of two kinds of features. On the global level the convex hull is used to extract the largest object in the image which is the fingerprint. On the local level, local mean and variance are used to segment the region within the convex hull and its border. The pre-processing step is invoked to enhance the fingerprint image and a post-processing stage is used to fill the gaps, if any, generated by the segmentation stage. The rest of the paper is organized as follows. In the next section the features for fingerprint segmentation state of the art is presented. In Section III the proposed algorithm is illustrated. The experimental results based on the proposed method are given in Section IV, and in Section V the conclusion is presented.

II. FEATURES FOR FRINGERPRINT SEGMENTATION Local mean and variance are the features for gray-level

based methods while coherence is the feature for the direction-based method. Coherence indicates the strength of the local window gradients centred on the processed point along the same dominant orientation. Local mean and variance are calculated as follows:

w

Iw

mean2

1 (1)

w

meanIw

Variance 22

)(1

(2)

Where I is the intensity and w is the window size cantered on the processed pixels. On the other hand, coherence [8] is calculated from the following equation:

)/()(4)( 22yyxxxyyyxx gggggcoh (3)

978-1-4673-2338-3/12/$31.00 ©2012 IEEE

w

xxx Gg 2 , w

yyy Gg 2 , yw

xxy GGg (4)

Where xG and yG are corresponding horizontal and vertical

gradient components which are given by Sobel operators. The background of a fingerprint image is the area where

the finger does not touch the sensor whereas the foreground is the area of the image where the ridges and the valleys are located. This means that background is the part of the image where bright pixels are located. This area is characterized by low variations in its brightness. Therefore, the mean in the background is higher than that of the foreground. In contrary, the variance is higher in the foreground parts compared with the background. To illustrate the effect of the mean and variance, image 1_4 in FVC DB1 [2] is divided into blocks and the mean and variance of each block is computed. A 3D plot of (x,y) coordinate of each block together with the corresponding mean is generated and depicted in Figure 1. The plot shows a good discrimination between the blocks belongs to the foreground compared with those belong to the background. Similar plots are generated for the invariance, coherence and the 3D distribution of the mean, variance and coherence of the blocks as shown in Figures 1. When the coherence is spatially plotted, it does not show a clear distinction between the foreground and background. By referring to the spatial plot of the coherence in Figure 1 and compare it with that of the variance, it is clear that the variance gives better separation of the foreground-background of the fingerprint image compared with the coherence. To generalise this issue, 40 different images are randomly selected from the FVC DB1, DB2, DB3, and DB4 and tested in the same manner. Results show that variance has better ability to separate the foreground from the background compared with the coherence. By definition coherence is used to examine the relationship between two datasets (in this case,

xxg and yyg ). If xxg and yyg are completely unrelated the

coherence will be zero. If its value is less than one but greater than zero, it is an indication that either the image is noisy or because the second dataset produces output due to first dataset as well as other inputs. In the case of fingerprint images, the logical explanation is that the set of images are noisy. On the other hand, the variance gives better discrimination because of the ridges and valleys nature of the fingerprint foreground.

III. THE PROPOSED APPROACH

In this paper, fingerprint segmentation is achieved in three stages:

1. Pre-processing Fingerprint images with low contrast, false traces ridges or noisy complex background cannot be segmented correctly. Therefore, such images should be enhanced. The only pre-processing applied in this paper is histogram stretching. This operation enhances the dynamic range of the image under consideration.

2. Segmentation

The proposed segmentation approach processes the fingerprint image in two levels. They are the global and the local levels. The global level:

In this level, the fingerprint image is coarse segmented into three different non-overlapped regions - foreground, background, and border such that the union of these three images makes the original image as shown in Equation 5:

BrBaFo ,,Im (5)

Where Im is the fingerprint image, Fo is the foreground region, Ba is the background region and Br is the border region.

Fig 1: Spatial distribution of fingerprint features. Row 1: image 1_4 of DB1-a

FVC2000. Row 2: Spatial distribution of mean (Left) and variance (Right), Row 3: coherence (Left) and Relationship among mean, variance and

coherence (Right).

Fig 2: Coarse segmentation by global processing.

Since the fingerprint foreground represents the object with the largest area in the image, locating this object in the image

means locating its foreground. Figure 2 depicts the steps followed to isolate this object in the image. The process starts by binarising the fingerprint image using the Otsu thresholding method [16] . To isolate the foreground from the rest of the image, dilation is applied to force the detached ridges to attach to each other. A square structuring element whose width is related to the size of the images under consideration is created. The fingerprint foreground becomes a large single object dominating the image. By applying a modified version of connected component labelling algorithm [17] and targeting the largest object in the image, the foreground of the fingerprint can be located and extracted. Extraction of this object takes place by localising the edges of the foreground area using Canny edge detector. Based on the edges defined in the former step, the foreground of the fingerprint image is separated from its background. The steps and results of this coarse segmentation are depicted in Figure 3. In order to locate the border of the fingerprint, the foreground image is divided into a number of non overlapped blocks of size wxw. The size of w is adaptive and its calculation is based on the number of pixels between the centre of one ridge and its neighbour.

Fig 3: Coarse segmentation. Upper row: (left) original image (right) largest

object.. Lower row: (left) edge detection. (right) separating foreground.

Consider a strip of one pixel width taken laterally or longitudinally anywhere in a fingerprint image where ridges and valleys exist. Plotting this strip gives a wave similar to that shown in Figure 4. In this plot, the x-axis represents the location of the pixels in the strip, while the y-axis is the gray level of each pixel. The maximum point of each cycle of the wave corresponds to a ridge while the minimum point corresponds to a valley. To calculate the wavelength of these cycles, a simple method is applied. The strip is first converted into black and white by applying a threshold. A single threshold will not give the desired result. Therefore, a wave is first treated by a regression technique called Locally Weighted Scatter-plot Smoothing (LOWESS)[18; 19]. The output of this

regression technique is a unique value of threshold calculated for each cycle of the wave (a ridge and a valley). Once this adaptive threshold is accomplished, the strip is converted into black and white, the centre of each ridge and valley is specified and the distance between the two neighbour ridges are computed. The minimum window size is then specified by the average of all distances between the centres of the ridges in the fingerprint image. Once the block size is specified, each block is classified into one of the sets mentioned above based on the following rules: 1. A block is assigned to the Fo set if all pixels in the wxw

window are located within the foreground area of the fingerprint.

2. A block is assigned to the Ba set if all pixels in the wxw window are located outside the foreground area of the fingerprint.

3. A block is assigned to the Br set if any number of pixels in the wxw window is located outside the foreground area of the fingerprint.

The output of this classification is the three images shown in Figure 5.

Fig 4 : Thresholding a fingerprint strip by LOWESS.

The local level: In this level, the fingerprint image is segmented by two

different sets of fuzzy rules. One set is for the Fo image and the other is for the Br image. The final segmented image is a combination of the two images. Features employed in this level are the mean and variance of the blocks. Two different fuzzy systems were created as depicted in Figure 6. The process of segmentation is implemented as follows: Both of the Fo and Br images are treated as wxw

blocks. A block in the Fo image is considered as

background if its local mean is high AND its local variance low. An AND operator is used here to minimize the number of blocks to be considered as

background, and hence maintain the foreground as one complete unit. if (mean is High) and (variance is Low) then (Block is Background).

A block in the Br image is considered as background if its local mean is high OR its local variance is low. The membership function is different from the former case because the border block contains a mixture of foreground and background pixels. However, an OR operator is used instead the AND operator to balance the effect of the high values in the fuzzy membership functions. if (mean is High) or (variance is Low) then (Block is Background).

The last step in this level is to combine the Fo and Br images to make the foreground image.

Fig 5: Coarse segmentation results. Row 1: (left) original image. (right) The

Fo image, Row 2:(left) The Br image. (right) The Ba image.

3. Post-processing The segmented fingerprint image may contain isolated background blocks which are surrounded by foreground blocks. These background blocks are foreground in the original image. A simple post-processing technique is proposed to eliminate the presence of these isolated blocks as illustrated in the following pseudo code: Pseudo code of the post-processing For all blocks in the image

o If a background block (i,j) is found If two or more neighbours in the N4

neighbourhood ((i, j-1), (i, j+1), (i-1, j) and (i+1, j)) blocks are foreground then change all of the pixel in the block (i,j) back to their original value before any segmentation. 

End if o End if 

End for

Mean

Variance

Output

Fig 6: Fuzzy memberships functions employed for local level segmentation.

After filling the foreground with the missing blocks, they are segmented individually by Otsu thresholding [16] approach. The algorithm works locally on the blocks of the image to find the corresponding optimal threshold for each block and segments the image into white background and black foreground. An example of an image which is treated with post-processing is depicted in Figure 7.

IV. EXPERIMENTAL RESULTS

A four-level segmentation scheme was proposed in previous research [20]. This scheme is based on the number of correctly segmented blocks in the image and is described as follows: Good, when more than 90% of the blocks are correctly

segmented. Almost Good, when 75% - 89% of the blocks are

correctly segmented. Almost Bad, when 60% - 74% of the blocks are correctly

segmented. Bad, when less than 60% of the blocks are correctly

segmented.

(a) (b)

(c) (d)

Fig 7: (a) Output of local feature level (b) Result of post-processing (c) Segmented image by local threshold (d) Applying thinning.

A number of images were selected from the FVC2000 DB1 and divided into a number of blocks. A human operator was asked to classify each block into a foreground or a background. These images were then classified by the current approach. This procedure was carried out for 20 images which were classified by [20] as good, almost good, almost bad, and bad. Results show that the maximum error is 7.11%, minimum error is 1.22%, mean error is 2.849% and standard deviation is 1.565. By this low error, all images from different schemes were classified by the current approach as good which indicates high robustness. The average number of misclassified blocks as false positives or false negatives was almost equal, calculated as 13 blocks in both cases. This equates to about 1.44% of the blocks which make up the image. Figure 8 depicts results achieved by the current approach compared with that achieved by [20].

Good

Almost good

Almost bad

Bad

Fig 8: A comparison with the algorithm described by [20]. Left-hand column: the original images. Middle column: segmentation from [20]. Right-hand

column: results from current approach.

The proposed approach was also tested on 100 fingerprint images which were selected randomly and without repetition from the FVC2000 database DB1, DB2, DB3, and DB4. These images were collected using two small-size and low-cost optical sensors. A number of segmented images are depicted in Figure 9.

Fig 9: Fingerprint images from FVC2000 database segmented by the proposed

approach.

V. CONCLUSIONS

The problem of fingerprint segmentation is one of the pattern classification paradigms which are not yet fully solved. This paper presents a new algorithm for the segmentation of low quality fingerprint images. It is achieved by using a combination of global and local processing. At the global level and based on the property that the fingerprint is the largest object in the image, the fingerprint is isolated from the image using a set of steps started by global thresholding followed by dilation, connected components labelling, ending with edge detection. At the local level, mean and variance of blocks are invoked to decide whether a certain block is a foreground or background using two sets of fuzzy rules. To evaluate this approach, a set of experiments were performed which showed the robustness of this approach. The mean error of block segmentation was not more than 2.84% while the false positives and false negatives are almost equal and equates to approximately 1.44% of the segmented blocks. The approach shows that it is able to segment 96% of images used for testing. All images which are segmented by [20] as almost good, almost bad and bad can now be segmented as good, which indicates the high robustness of this approach.

REFERENCES [1] M.U. Akram, S. Nasir, A. Tariq, I. Zafar, W.S. Khan, Improved

Fingerprint Image Segmentation Using New Modified Gradient Based Technique, in, 2008 Canadian Conference on Electrical and Computer Engineering, Niagara Falls, Canada, 2008, pp. 1967 - 1972

[2] D. Maio, D. Maltoni, A. Jain, S. Prabhakar, Handbook of fingerprint recognition second ed., Springer, London, 2009.

[3] J. Yin, E. Zhu, X. Yang, G. Zhang, C. Hu, Two steps for fingerprint segmentation. Image and Vision Comput. 25 (2007) 1391-1403.

[4] B. Weixin, X. Deqin, Z. Yi-wei, Fingerprint Segmentation Based on Improved Active Contour, in, IEEE Computer Society International Conference on Networking and Digital Society,, 2009, pp. 44-47.

[5] W. Feng, W. Xiuyou, X. Lin, An Improved Fingerprint Segmentation Algorithm Based on Mean and Variance, in, 2009

International Workshop on Intelligent Systems and Applications, Wuhan, China, 2009, pp. 1-4.

[6] C. Yu, M. Xie, J. Qi, An Effective Algorithm for Low Quality Fingerprint Segmentation, in, IEEE 3rd International Conference on Intelligent System and Knowledge Engineering, Chengdu, China, 2008, pp. 1081-1085.

[7] M. Helfroush, M. Mohammadpour, Fingerprint segmentation, in, 3rd International Conference on Information and Communication Technologies: From Theory to Applications, Damascus, Syria, 2008.

[8] A.M. Bazen, S.H. Gerez, Segmentation of Fingerprint Images, in, ProRISC 2001 Workshop on Circuits, Systems and Signal Processing, Veldhoven, The Netherlands, 2001, pp. 276-280.

[9] C. Ren, Y. Yin, J. Ma, G. Yang, A Linear Hybrid Classifier for Fingerprint Segmentation, in, IEEE Fourth International Conference on Natural Computation, Jinan, China, 2008, pp. 33-37.

[10] H. Ailisto, M. Lindholm, A review of fingerprint image enhancement methods. International Journal of Image and Graphics 3 (2003) 401-424.

[11] A. Tahmasebi, S. Kasaei, A novel adaptive approach to fingerprint enhancement filter design. Signal Process. Image Commun 17 (2002) 849-855.

[12] E. Yun, S. Cho, Adaptive fingerprint image enhancement with fingerprint image quality analysis. Image Vision Comput. 24 (2006) 101-110.

[13] E. Zhu, J. Yin, C. Hu, G. Zhang, A systematic method for fingerprint ridge orientation estimation and image segmentation. Pattern Recognition 39 (2006) 1452-1472.

[14] P.B. Marques, G. Thome, A Neural Network Fingerprint Segmentation Method, in, IEEE Fifth International Conference on Hybrid Intelligent Systems, Rio de Janeiro, Brazil, 2005, pp. 385-392.

[15] J. Kang, W. Zhang, Fingerprint Image Segmentation using Modified Fuzzy C-Means Algorithm, in, 3rd International Conference on Bioinformatics and Biomedical Engineering, Beijing, China, 2009.

[16] N. Otsu, A threshold selection method from gray level histogram. IEEE Trans. Syst.Man Cybern SMC-9 (1979) 62-66.

[17] K. Suzuki, I. Horiba, N. Sugie, Linear-time connected component labelling based on sequential local operations. Computer Vision and Image Understanding 89 (2003) 1-23.

[18] W. Cleveland, Robust Locally Weighted Regression and Smoothing Scatterplots. Journal of the American Statistical Association 74 (1979) 829-836.

[19] W. Cleveland, S. Devlin, Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting. Journal of the American Statistical Association 83 (1988) 596-610.

[20] H. Fleyeh, D. Jomaa, Segmentation of Low Quality Fingerprint Images, in, IEEE International Conference on Multimedia Computing and Information Technology (MCIT-2010), Sharja - UAE, 2010, pp. 85-88.