Separating Overlapped Fingerprints Using Constrained Relaxation Labeling

Yuan Shi, Jianjiang Feng, Jie ZhouDepartment of Automation

Tsinghua University, Beijing, China{jfeng,jzhou}@tsinghua.edu.cn

Abstract

Overlapped fingerprints are frequently encountered inlatent fingerprints lifted from crime scenes. It is necessaryto separate such overlapped fingerprints into componentfingerprints so that existing fingerprint matchers can recog-nize them. The most crucial step in separating overlappedfingerprints is the separation of mixed orientation fields intocomponent orientation fields. In this paper, a constrainedrelaxation labeling algorithm is proposed to perform theorientation field separation task. Experimental results onboth real and simulated overlapped fingerprints show thatthe proposed algorithm outperforms the state-of-the-art al-gorithm in both accuracy and efficiency.

1. IntroductionFingerprint is the most widely used trait in biometric

recognition. Thanks to its uniqueness and persistence, fin-gerprint recognition has been successfully deployed in vari-ous applications, such as entry control, time and attendance,computer login, forensics, and airport security. Althoughfingerprint recognition technology has advanced rapidly inthe past fourty years [7], there are still some challengingresearch problems. One challenging problem that has re-ceived little attention is the processing and matching ofoverlapped fingerprints.

Overlapped fingerprints are frequently encountered in la-tent fingerprints lifted from crime scenes. Since existingfingerprint feature extraction algorithms are developed un-der the assumption that fingerprint images contains onlyone fingerprint, they cannot correctly process overlappedfingerprints. See Fig. 1 for the poor result of applying awell-known commercial fingerprint algorithm, VeriFinger6.2 SDK, to an overlapped fingerprint image. To identifyoverlapped fingerprints, fingerprint examiners have to care-fully mark the minutiae of each overlapping fingerprint sep-arately. This process is very time-consuming, tedious andprone to error. It is desired to develop an algorithm to sepa-rate the overlapped fingerprints into individual fingerprints

Figure 1. The skeleton images extracted from an overlapped fin-gerprint image by the well known commercial software, VeriFin-ger 6.2 SDK.There are many missing and spurious ridges in theoverlapped region of the skeleton images.

in order to reduce the labor of fingerprint examiners.Only a few studies are related to separating overlapped

fingerprints [4, 3, 10, 2]. Among them, the work by Chenet al. [2] may be viewed as the state of the art. In [2], byapplying a relaxation labeling method on the initial/mixedorientation fields obtained by local Fourier analysis, the twocomponent orientation fields are obtained. The two com-ponent fingerprints are then separated by filtering the over-lapped fingerprint image using Gabor filters tuned to thecomponent orientation fields. In this method, the most cru-cial step is to separate the mixed orientation field into com-ponent orientation fields.

In this paper, we propose an overlapped orientation fieldseparating method based on constrained relaxation labeling.This method differs from the algorithm in [2] in the follow-ing aspects:

1. Utilization of non-overlapped area. In this work, non-overlapped area is utilized as important constraintsduring the relaxation labeling process, while in [2], un-til the relaxation labeling is finished, non-overlappedarea is then simply combined with consistent compo-nent orientation fields. Using non-overlapped area as

978-1-4577-1359-0/11/$26.00 ©2011 IEEE

Figure 2. Flowchart of the proposed separation algorithm.

constraints not only leads to better separation results,but also speeds up the convergence of relaxation label-ing.

2. Mutual exclusion constraint. We treat each overlappedblock as a single object rather than two ones to strictlyenforce the mutual exclusion constraint, namely, twocandidate orientations in an overlapped block cannotbelong to the same fingerprint. In [2], however, thisconstraint is only loosely used because the two candi-date orientations in an overlapped block are treated astwo objects.

3. Order of updating label probabilities. We sequentiallyupdate label probabilities in an overlapped block in anascending order of the distance between itself and non-overlapped area. While in [2], the label probabilities inall overlapped blocks are updated in parallel. Sequen-tial updating is beneficial for both separation perfor-mance and computational complexity.

The proposed overlapped fingerprint separation algo-rithm consists of the following steps (see the flowchart inFig. 2):

1. Region segmentation: The overlapped fingerprint im-age is divided into background region, overlapped re-gion, non-overlapped regions of two component fin-gerprints.

2. Initial orientation field estimation: One dominant ori-entation is estimated in the non-overlapped regionwhile two dominant orientations are estimated in theoverlapped region.

3. Orientation field separation: The initial orientationfield is separated into two component orientationfields.

4. Fingerprint separation: Two component fingerprintsare obtained by filtering the overlapped image withGabor filters tuned to the two component orientationfields respectively.

In section 2, we described the steps 1, 2, and 4. Step 3,which is the most crucial step and also the major contribu-tion of this paper, is described in section 3. Section 4 showsthe results of experiments. Finally we present the conclu-sions and future work in section 5.

2. Overlapped Fingerprint Separation

2.1. Region Segmentation

In this paper, we consider the situation where the over-lapped fingerprints image contains only two component fin-gerprints. The masks of the two component fingerprints aremarked manually. An overlapped fingerprint image consists

Figure 3. Dividing the overlapped fingerprint image into differentregions.

of three regions: the background region, the overlapped re-gion, and the non-overlapped region of two component fin-gerprints. The background region does not contain ridgepatterns. The overlapped region is the common region ofthe two masks (i.e. the intersection of the two masks) and itcontains the overlapped part of the two fingerprints. Thenon-overlapped region contains only one fingerprint (i.e.the subtraction of the two masks). Fig. 3 gives the pro-cedure of dividing regions.

An overlapped fingerprint image is segmented into non-overlapping blocks of 16×16 pixels. The block in the over-lapped region is called overlapped block and the block in thenon-overlapped region is called non-overlapped block. Weestimate one dominant orientation in each non-overlappedblock and two dominant orientations in each overlappedblock.

2.2. Initial Orientation Field Estimation

Since the non-overlapped region contains only one fin-gerprint, traditional orientation field estimation methods,such as gradient-based [1] or slit-based [8], can be usedto estimate the orientation in each non-overlapped block.But in the overlapped region, traditional methods will fail.Considering the overlapped block as two groups of stripesof different directions, we use the local Fourier analysismethod [6] instead. Discrete Fourier Transform is calcu-lated in the window of 64× 64 pixels with each overlappedblock in its center. The bright points (i.e. local maximapoints) in the frequency spectrum correspond to stripes ofdifferent directions. The two brightest points (i.e. two lo-cal maxima points of largest amplitude) correspond to thetwo orientations of each overlapped block. We only searchthe bright points in the area between the two red circles inthe frequency domain, which correspond to valid ridge fre-quency (See Figure 4).

It is worth noting that the directions of the two stripes

Figure 4. Estimation of two dominant orientations in an over-lapped block.

may be the same. To handle this case, we compute the ratioof the amplitude of the first brightest point to the amplitudeof the second brightest point. If the ratio is lager than athreshold (i.e. the second brightest point is spurious), theorientations of the two stripes are deemed as the same andthe first brightest point corresponds to the two stripes.

2.3. Fingerprint Separation

After orientation estimation, constrained relaxation la-beling will be used to determine the labels of orientationsof overlapped blocks. Constrained relaxation labeling willbe described in detail in the next section. Here we introducethe steps of how to separate the overlapped fingerprints afterconstrained relaxation labeling.

After constrained relaxation labeling, the orientationfields of the two component fingerprints are obtained. Butsome errors may occur in some area of the orientation field.An error correction algorithm is utilized to smooth the ori-entation field. The block orientation in each overlappedblock is modified as the mean orientation of all block orien-tations in the 5 × 5 block-wise square area centered at theblock being considered.

After the two component orientation fields are smoothed,the two block-wise frequency images are calculated usingthe x-signature method in [5]. Finally, two component fin-gerprints are obtained by filtering the overlapped fingerprintimage with Gabor filters tuned to the corresponding orien-tation field and frequency image [5].

3. Constrained Relaxation Labeling3.1. Problem Statement

After initial orientation estimation, we obtain two ori-entations in each overlapped block. But we do not knowtheir labels (i.e. which orientation belongs to which finger-print). Relaxation labeling [9] is used to identify the labelsof the two orientations in [2]. But their result is not satisfac-tory because the authors only use the orientations in eachoverlapped block and do not take the inherent constraintsof them into account. In this paper we propose the con-

strained relaxation labeling algorithm to identify the labels.Not only the orientations in overlapped blocks but also theorientations in non-overlapped blocks are taken into consid-eration.

We introduce a two dimension vector to denote thetwo orientations of each overlapped block. Suppose thereare N blocks in overlapped region and the vector ai =( ai1 ai2 )T is the orientation-vector of the ith over-lapped block (ai1 and ai2 are the two orientations). SoA = {a1, a2, . . . , aN} is the set of N orientation-vectorobjects to be labeled. We use the set L = {1, 2} to describethe labels of the orientations. Since ai is two dimensionvector, we introduce a state set S = {1, 2} to describe thestate of labels of ai1 and ai2:

sai =

{1 if label(ai1) = 1, label(ai2) = 22 if label(ai1) = 2, label(ai2) = 1.

(1)

The orientations of the non-overlapped region #1 aregiven label 1 and the orientations of the non-overlapped re-gion #2 are given label 2. Suppose there are M blocks inthe two non-overlapped regions and bj is the orientation ofthe jth non-overlapped block. So B = {b1, b2, . . . , bM}is the set of M orientations whose labels are known. Tokeep correspondence with A, we also introduce the state setS = {1, 2} to describe the label of bj :

sbj =

{1 if label(bj) = 12 if label(bj) = 2.

(2)

Now A ∪ B = {a1, a2, . . . , aN , b1, b2, . . . , bM} is theset of the orientations of blocks in overlapped region andnon-overlapped region.

Let pai(sai) be the probability that the state sai is thecorrect state of and satisfy: 0 ≤ pai(sai) ≤ 1 and pai(1) +pai(2) = 1. Let pbj (sbj ) be the probability that the statesbj is the correct state of bj and satisfy: 0 ≤ pbj (sbj ) ≤ 1and pbj (1) + pbj (2) = 1. Because the states of all non-overlapped blocks are known, we find that the state proba-bility of bj is pbj (1) = 1, pbj (2) = 0 if bj belongs to thefirst non-overlapped region and pbj (1) = 0, pbj (2) = 1, ifbj belongs to the second non-overlapped region.

3.2. Compatibility Function

In relaxation labeling, the labels of objects affect eachother. The compatibility function is used to measure thecorrelation between the labels of two objects. In this paper,the states of two objects in A ∪ B affect each other. Weconsider two types of compatibility functions: one for twoobjects in A, and another one for one object in A and oneobject in B.

The first type of compatibility function is to measure theaffection from ai in A to aj in A. We can denote this type of

compatibility function in the form of conditional probabil-ity: Caa

ij (sai , saj ) = pai,aj (saj |sai). Because of symmetrywe can find that:{

Caaij (1, 1) = Caa

ij (2, 2) = βCaa

ij (1, 2) = Caaij (2, 1) = 1− β,

(3)

whereβ =

disaa12disaa11 + disaa12

disaa11 = |ai1 − aj1|+ |ai2 − aj2|

disaa12 = |ai1 − aj2|+ |ai2 − aj1|.

The compatibility is measured according to the 1-norm dis-tances of different states of ai and aj . The larger the dis-tance is, the lower the compatibility is.

The second type of compatibility function is to measurethe affection from bk in B to aj in A. We can define thistype of compatibility function like this: Cba

kj (sbk , saj ) =pbk,aj (saj |sbk). Because of symmetry we can find that:{

Cbakj (1, 1) = Cba

kj (2, 2) = λ

Cbakj (1, 2) = Cba

kj (2, 1) = 1− λ,(4)

where

λ =disba12

disba11 + disba12

disba11 = |bk − aj1|

disba12 = |bk − aj2|.

The second type of compatibility is also measured accord-ing to the distances of different states of bk and aj . Thedistance here has a special form.

Since the state of arbitrary bj is known, we do not needto define the compatibility function from arbitrary bi in Bto bj in B or from arbitrary ai in A to bj in B.

3.3. State Probability Updating

Relaxation labeling is an iterative procedure. In each it-eration, the state probability of aj will be updated by con-sidering all the state probability of any neighboring objectai and bk (if ai’s block is the neighbor of aj’s block, we callai the neighbor of aj ; so is bk).

We can calculate the probability increment like this:

∆paj(saj

) =∑

ai∈A,i ̸=j

waaij

∑sai

∈S

pai(sai

)Caaij (sai

, saj)+∑

bk∈B

wbakj

∑sbk∈S

pbk(sbk)Cbakj (sbk , saj ),

(5)where waa

ij is the weight used to measure how strongly ai af-fects aj and wba

kj is the weight used to measure how stronglybk affects aj . We only take into account the blocks in thesquare 5 × 5 block-wise area whose center is aj’s block.

Figure 5. Updating order in constrained relaxation labeling. Thestate probabilities in overlapped blocks are updated in the orderindicated by the red arrow.

The weights of the blocks outside the 5× 5 block-wise areais set to 0. Inside the square area, waa

ij is related to the dis-tance from ai’s block to aj’s block, and wba

kj is related to thedistance from bk’s block to aj’s block.

In each iteration, we do not change any pbk(sbk), sincethe state probability of any bk in B is known and fixed. Weupdate paj (saj ) using the following equation:

pnewaj(saj ) =

poldaj(saj )(1 + ∆paj (saj ))∑

saj∈S

poldaj(saj )(1 + ∆paj (saj ))

. (6)

Generally, we are more certain about the label probabil-ity of those overlapped blocks which are close to the non-overlapped blocks. Thus we first update the probability ofoverlapped blocks adjacent to the non-overlapped region,then those overlapped blocks (which are not yet updated)adjacent to the updated overlapped blocks, and so on. Theupdating order looks like a spiral pattern (see Fig. 5). Whilein [2], the label probabilities are updated in parallel becauseonly overlapped blocks are taken into account in relaxationlabeling and the label probabilities of all overlapped blocksare equally uncertain.

We update all paj (saj ) in one iteration. The iteration isrepeated until all paj (saj ) converge or the maximum num-ber of iterations is exceeded.

3.4. Orientation Field Separation

After the constrained relaxation labeling process, we ob-tain the final state probability paj (saj ). The state whoseprobability is larger is determined as the right state of aj .And the labels of two orientations aj1 and aj2 are deter-

mined:{label(aj1) = 1, label(aj2) = 2 if paj (1) ≥ paj (2)label(aj1) = 2, label(aj2) = 1 if paj (1) < paj (2).

(7)Because the label of any bk is known and fixed, we now

obtain the orientation fields of the #1 and #2 fingerprintsin the overlapped fingerprint image. The orientation fieldof the #1 fingerprint consists of block orientations whoselabel is 1 and the #2 orientation field is composed of theblock orientations with label 2. After the two componentorientation fields are obtained, we use the method describedin section 2.3 to separate the two component fingerprints.

Fig. 6 shows the separated fingerprints from four latentoverlapped fingerprint images via the proposed method andthe method in [2]. We can observe that the proposed methodprovides a better separation result than the method in [2].

4. Experimental Results

The final goal of separating overlapped fingerprint im-ages is to successfully match the component fingerprint tothe corresponding template fingerprint. We utilize the samedata set which was used in [2] to test the performance ofour separating method. As recited in [2], the images of over-lapped fingerprints are synthesized by the no. 3 impressionsand no. 4 impressions of ten fingers in FVC2002 DB1 B.Each no. 3 impression will be overlapped with no. 4 im-pressions of all the ten fingers. So there are 10× 10 = 100images of overlapped fingerprints. No. 1, 2, 5, 6, 7 and 8impressions of the ten fingers are utilized as template fin-gerprints which are matched with the separated fingerprintsby using VeriFinger matcher. Only genuine matches (i.e.match the testing fingerprint image and template fingerprintimages belonging to the same finger) are executed becausethe output scores of the VeriFinger matcher are linked to theFalse Accept Rate (FAR).

In order to compare the two algorithms, four groups ofmatching experiments are conducted. In the first group(called ideal separation), no. 3 and 4 impressions arematched with the template fingerprints, which gives theupper bound of the matching accuracy. In the secondgroup (called no separation), the overlapped fingerprint im-ages (segmented using the two region masks) are directlymatched with the template fingerprints, which gives thelower bound of the matching accuracy. In the third andfourth groups, our method and Chen et al.’s method (us-ing singularity information) are used to separate the over-lapped images respectively, and the separated fingerprintsare matched with the template fingerprints. The left fourcolumns in Fig. 7 show the input images of four groups ofmatching experiments for an overlapped image. The Re-ceiver Operating Characteristic (ROC) curves are given inFig. 8. We observed that at the same FAR, our method’s

Figure 6. Some examples of the separated fingerprints from latent overlapped fingerprints. Each row shows a group of separating results.In each group, the image in first column (from left to right) is the latent overlapped fingerprint image; the second and third columns areseparated fingerprints by the proposed method; the fourth and fifth columns are separated fingerprints by Chen et al.’s method.

True Accept Rate (TAR) is at least 10 percent higher thanthe TAR of the method in [2].

There is still room for improvement in our method. Inthe experiments, we find that the estimation error of the ini-tial orientations in the overlapped blocks greatly impact theseparation results. The smaller the estimation error is, thebetter the quality of the separated fingerprints will be, andthe matching accuracy will be higher. In order to demon-strate this, we conduct two groups of contrast experiments.In the first group of experiments, we use the ideal orien-tation fields (we obtain this information from the originalfingerprint image) of the two component fingerprints to sep-arate the overlapped fingerprints. In the second experiment,

we use the true values of the two orientations in the over-lapped blocks as the two estimated value (i.e., the initialorientation field is ideal), but their labels are not known andwe use the proposed relaxation labeling algorithm to deter-mine them. The other parts of the separation algorithm arethe same in all experiments. The input images for these twoexperiments are given in the fifth and sixth columns in Fig.7. The ROC curves of these two groups of matching ex-periments and the ROC curves of ideal separation and theproposed separation algorithm are shown in Fig. 9. As canbe observed from this figure, the performance of our methodusing ideal initial orientation field is almost the same as theperformance of directly using the ideal orientation fields.

Figure 7. Input images of six groups of matching experiments. Column 1: original component fingerprints (i.e. obtained by ideal separa-tion); Column 2: overlapped fingerprints (i.e. no separation); Column 3: component fingerprints separated by Chen et al.’s method; Column4: component fingerprints separated by the proposed method; Column 5: component fingerprints separated using ideal orientation fieldswhich are estimated from the original component fingerprints and whose labels are known; Column 6: component fingerprints separatedby the proposed method using ideal initial orientation field which are estimated from the original component fingerprints but whose labelsare unknown.

Figure 8. ROC curves of four groups of matching experimentsusing different input images: (1) original component fingerprints(i.e. obtained by ideal separation), (2) overlapped fingerprints (i.e.no separation), (3) component fingerprints separated by Chen etal.’s method, and (4) component fingerprints separated by the pro-posed separation method.

This indicates that the proposed constrained relaxation la-beling algorithm is satisfactory, while the initial orientationfield estimation may need large improvement.

The proposed method is much faster than Chen et al.’smethod. The average time of the proposed constrainedrelaxation labeling algorithm is about 3 seconds, whileit is 36 seconds for Chen et al.’s method. The otherparts of the two separation algorithms are very similar inspeed. It is worth noting that the proposed method achieveshigher accuracy without using the information of singu-

Figure 9. ROC curves of four groups of matching experimentsusing different input images: (1) original component fingerprints(i.e. obtained by ideal separation), (2) component fingerprints sep-arated by using ideal orientation fields, (3) component fingerprintsseparated by the proposed separation method using ideal initialorientation field, and (4) component fingerprints separated by theproposed separation method.

lar points, while manually marked singular points are usedin Chen et al.’s method. The Matlab implementation ofthe proposed algorithm has been made available for com-parison purpose at http://ivg.au.tsinghua.edu.cn/people/˜JianjiangFeng/software.html.

5. Conclusions and Future WorkSeparating overlapped fingerprints into component fin-

gerprints is very useful in latent fingerprint recognition.

We proposed an algorithm for separating overlapped finger-prints, which outperforms the state of the art method in bothaccuracy and efficiency. In addition, our method does notrequire the information of singular points, and thus costsless human labor.

There is still limitations in our method. Initial orienta-tions in the overlapped blocks have a large influence on theseparation performance. As a future work, we will explorehow to estimate the initial orientation more accurately. Thecurrent separation algorithm is not yet fully automatic. Theregion masks of the two components fingerprints must bemarked manually. As another future work, we plan to de-velop a fully automatic separating algorithm which can es-timate the region masks of component fingerprints automat-ically.

Acknowledgments

This work was supported by the National Natural Sci-ence Foundation of China under Grants 61020106004,60875017, 61005023, and 61021063.

References[1] A. M. Bazen and S. H. Gerez. Systematic Methods for the

Computation of the Directional Fields and Singular Pointsof Fingerprints. IEEE Transactions on Pattern Analysis andMachine Intelligence, 24(7):905–919, 2002.

[2] F. Chen, J. Feng, A. K. Jain, J. Zhou, and J. Zhang. Sepa-rating Overlapped Fingerprints. IEEE Transactions on Infor-mation Forensics and Security, 6(2):346–359, 2011.

[3] X. Fan, D. Liang, and L. Zhao. A scheme for separatingoverlapped fingerprints based on partition mask. ComputerEngineering and Applications, 40(2):80–81, 2004 (in Chi-nese).

[4] R. Geng, Q. Lian, and M. Sun. Fingerprint separation basedon morphological component analysis. Computer Engineer-ing and Applications, 44(16):188–190, 2008 (in Chinese).

[5] L. Hong, Y. Wan, and A. K. Jain. Fingerprint Image En-hancement: Algorithm and Performance Evaluation. IEEETransactions on Pattern Analysis and Machine Intelligence,20(8):777–789, 1998.

[6] A. K. Jain and J. Feng. Latent Palmprint Matching. IEEETransactions on Pattern Analysis and Machine Intelligence,31(6):1032–1047, 2009.

[7] D. Maltoni, D. Maio, A. K. Jain, and S. Prabhakar. Hand-book of Fingerprint Recognition. Springer-Verlag, secondedition, 2009.

[8] M. A. Oliveira and N. J. Leite. A Multiscale DirectionalOperator and Morphological Tools for Reconnecting Bro-ken Ridges in Fingerprint Images. Pattern Recognition,41(1):367–377, 2008.

[9] A. Rosenfeld, R. A. Hummel, and S. W. Zucker. Scene label-ing by relaxation operations. IEEE Transactions on Systems,Man and Cybernetics, 6(6):420–433, 1976.

[10] M. Singh, D. Singh, and P. Kalra. Fingeprint separation: anapplication of ICA. In Proceedings of the SPIE, Mobile Mul-timedia/Image Processing, Security, and Applications, vol-ume 6982, pages 69820L–1–69820L–11, 2008.