spectral optimized asymmetric segmented phase-only
TRANSCRIPT
HAL Id: hal-00782966https://hal.archives-ouvertes.fr/hal-00782966
Submitted on 31 Jan 2013
HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.
Spectral optimized asymmetric segmented phase-onlycorrelation filter
Isabelle Leonard, Ayman Alfalou, Christian Brosseau
To cite this version:Isabelle Leonard, Ayman Alfalou, Christian Brosseau. Spectral optimized asymmetric segmentedphase-only correlation filter. Applied optics, Optical Society of America, 2012, 51, pp.2638-2650.�hal-00782966�
1
Spectral optimized asymmetric segmented phase-only
correlation filter
I. Leonard1, A. Alfalou,
1 and C. Brosseau
2
1 ISEN Brest, Département Vision, L@bISEN,
20 rue Cuirassé Bretagne, CS 42807, 29228 Brest Cedex 2, France
E-mail: [email protected]
2 Université Européenne de Bretagne, Université de Brest, Lab-STICC,
CS 93837, 6 avenue Le Gorgeu, 29238 Brest Cedex 3, France
E-mail: [email protected]
OCIS codes: 070.0070, 100.3008, 100.5010, 100.5090.
2
Abstract
We suggest a new type of optimized composite filter, i.e. the asymmetric segmented
phase-only filter (ASPOF), for improving the effectiveness of a VanderLugt
correlator (VLC) when used for face identification. Basically, it consists in merging
several reference images after application of a specific spectral optimization method.
After segmentation of the spectral filter plane to several areas, each area is assigned
to a single winner reference according to a new optimized criterion. The point of the
paper is to show that this method offers a significant performance improvement on
standard composite filters for face identification. We first briefly revisit composite
filters (adapted, phase-only, inverse, compromise optimal, segmented, minimum
average correlation energy, optimal trade-off maximum average correlation, and
amplitude modulated phase-only (AMPOF)) which are tools of choice for face
recognition based on correlation techniques and compare their performances with
those of the ASPOF. We illustrate some of the drawbacks of current filters for
several binary and gray scale images identifications. Next, we describe the
optimization steps and introduce the ASPOF that can overcome these technical issues
to improve the quality and the reliability of the correlation based decision. We derive
performance measures, i.e. peak-to-correlation energy values and receiver operating
characteristic curves, to confirm consistency of the results. We numerically find that
this filter increases the recognition rate and decrease the false alarm rate. The results
show that the discrimination of the ASPOF is comparable to that of the AMPOF, but
the ASPOF is more robust than the trade-off maximum average correlation height
3
(OT-MACH) against rotation and various types of noise sources. Our method has
several features that make it amenable to experimental implementation using a VLC.
4
1. Introduction Intense interest in optical correlation techniques over a prolonged period has focused
substantially on the filter designs. These techniques represent a powerful tool for a wide range of
applications like target tracking, identification, and classification. These applications require a
real time processing, must be robust to several perturbations, e.g. face rotation, and should be
insensitive to noise. The optical correlation technique is a good candidate for very fast
recognition systems. However, the principle of correlation based on a comparison between the
target image and a correlation filter makes this method not robust to rotation. One of the
techniques used to overcome this problem is the composite filter, i.e. it contains information
coming from multiple images. Based on the composite technique, the overall goal of this line of
research is to develop and optimize a novel filter for the VanderLugt correlator used in face
recognition [1] that is able to fulfill the above requirements and can handle a large database. In
this study, we use different faces with several viewpoints from the Pointing Head Pose Image
Database (PHPID) [2]. The results will show the impact of using the fusion criterion permitting
to assign a single reference to each filter-pixel. Moreover, thanks to the symmetry property of the
spectrum the number of real references in the composite filter is increased. This has for effect to
improve significantly the decision making process.
Following this brief introduction, we have divided the rest of the paper as follows: a general
overview of optical correlation methods is given in Sec. 2 providing technical details for the
system under consideration, and a connection to earlier ideas. In Sec. 3, we present a number of
relevant examples to illustrate our strategy for improving the effectiveness of the VLC using the
asymmetric segmented phase-only filter (ASPOF). This face recognition algorithm was
compared to a whole set of composite correlation filters: all systems were trained and tested on
5
the same images. Our conclusion is that the increase of performance obtained with the ASPOF
indicates that we were able to propose and validate a new composite correlation filter allowing
us to increase considerably the number of references to be incorporated in one filter. The ROC
curves permitted to distinguish between the good recognition and the true non recognition values
based on an optimized PCE criterion. Finally, we draw our conclusions in Sec. 4.
2. Some preliminary considerations and relation to previous work
The optical correlation method can be traced back to the pioneering research in the 1960s [1,3].
This method consists of comparing a target image and a reference image that comes from a
database. This comparison leads to correlation peak in the correlation plane. This correlation
peak is more or less intense, depending on the degree of resemblance between the target and
reference images. To optimize recognition, several filters created from the reference image have
been studied [4-6]. However, the correlation approach which needs to compare the target image
with a large number of reference images in order to increase its robustness is time consuming. To
reduce the processing time, several composite filters have been developed [6-11].
Over the last decade, there has been a resurgence of interest, driven by recognition and
identification applications [12-17], of the correlation methods. For example, Alam et al. [17]
demonstrated the good performances of the correlation method compared to all numerical ones
based on the independent component model. Another significant example in this area of research
is the work by Romdhani et al. [18], which compared face recognition algorithms with respect to
those based on correlation. These references are far from a complete list of important advances,
but fortunately the interested reader can easily trace the historical evolution of these ideas with
Vijaya Kumar's review paper, Yu’s book, and the chapter of Alfalou and Brosseau containing an
6
extensive bibliography [4, 6-7, 19]. According to the above considerations, our goal is to propose
and validate a new robust and discriminating composite correlation filter which is able to include
a large set of training images into this filter which allows one to deal with the diversity of facial
expressions as input information. To proceed further, we begin with a brief overview of the VLC
method. Next, we wish to specify the definition of standard performance measures that will be
used to compare filters.
2.1. VLC
In this study we consider only the VLC scheme and we focus on the design of a new type of
optimized composite filter. VLC was preferred to the joint transform correlator (JTC) [20-23]
because it allows exploitation of all the finite bandwidth in the input plane [24]. Fig. (1)
illustrates a simplified schematic of the VLC [1,4]. It is based on the multiplication between the
spectrum, obtained by Fourier transforming the target image (Fig 2 (a)), with a correlation filter
made from the FT of a reference image and placed in the Fourier plane. The correlation plane is
obtained from FT-1
of the product of both spectra. This results in a more or less intense central
correlation peak depending on the degree of similarity between the target object and the image
reference. Many approaches for designing the correlation filter can be found in the literature
according to the specific objects that need to be recognized [4, 6, 19, 25].
2.2. Performance measures for filter design
In order to quantify the correlation plane by the correlation of a filter and a test image different
correlation plane performance measures were used [26-27]. We choose the PCE criterion [26-27]
which is defined by
7
PCE = (Energy in the correlation peak)/(overall energy in the correlation plane) (1)
In other words, the PCE must be interpreted as the weight of the output signal (correlation peak)
in the correlation plane relative to noise. In practice, one should take into account of the width of
the correlation peak in the calculation of the PCE. In what follows, PCE will be evaluated as
0 0
0 0
2 2
1 1
PC E , ,
x x t y y t y Mx N
x x t y y t x y
C x y C x y
(2)
where (x0,y0) denote the position of the correlation peak, C(x,y) denotes the value of the
correlation plane at point (x,y), t is set to the number of neighboring pixels used, and (N,M)
denote the size (in pixels) of the correlation plane. A distinct advantage of the PCE is that it
permits evaluation of the relative importance of the correlation peak with reference to the noise
of the correlation plane.
Among standard measures for parametrizing the correlator performances, the receiver
operating characteristic (ROC) curve, which allow characterizing the binary classifiers, was
selected [28]. The ROC curve can be represented by plotting the fraction of true positives out of
the positives TPR (sensitivity, or true positive rate) versus the fraction of false positives out of
the negatives FPR (1- specificity, or false positive rate). We find it can be more useful to
consider adapted ROC curves [28]. We fix a weight according to the relative position of the PCE
with respect to threshold. We shall consider 10 zones separating the threshold and the maximum
PCE values obtained for a target face, and affect the weight =0.1 for the first zone, 0.2 to the
second zone, and so on.
8
3. Asymmetric segmented composite POF (ASPOF): a spectral
optimized filter The preceding analysis demonstrates the impact of a reduced number of correlations to obtain a
reliable decision by using a composite filter. However, this filter needs to be optimized in order it
can permits to merge a large number of reference images within a given filter. For that purpose
we suggest a new composite filter, the ASPOF. The ASPOF is an optimized version of the
segmented correlation filter [8]. Our algorithm has been extensively tested by comparing its
performance with those of several composite filters [30].
3.1. Segmented composite filter (SPOF)
The main idea behind the SPOF is that the high saturation regions of the reference images are
suppressed. Briefly stated, this is achieved through two steps [8]. First, a segmentation of the
spectral plane of the correlation filter is realized into several independent regions. Second, each
region is assigned to a single reference. This assignment is done according a specific energy
criterion
, ,
, ,
, ,
l k
u v u v
l k
i j i j
i j i j
E E
E E
(3)
where Elu,v denotes the energy on the pixel (u,v) of the reference image (l) in the spectral domain.
This criterion compares the energy (normalized by the total energy of the spectrum) for each
frequency of a given reference with the corresponding energies of another reference. Assignment
of a region to one of the two references is done according Eq. (3). Hence, the SPOF contains
frequencies with the largest energy.
9
3.2. ASPOF algorithm
A serious limitation of the SPOF is that the regions to which are assigned the winning references
become smaller as the similarity between the reference images is increased. This effect
eventually leads to isolated pixels. This issue can be critical since this paper deals with training
sets of reference images bearing a strong resemblance, i.e. data-base obtained by rotating images
within the range (-90°,+90°), and we want to increase the number of reference images in the
filter. Our optimizing method follows from the four-step scheme which defines the ASPOF.
(i) We first separate the reference image base ( Ni
R...1
, where N denotes the number of
used references), in two sub-classes: one, 1
iR , which contains the reference images with
1,...,5,3,1 Ni , the other one, 2
iR , deal with the other indices Ni ,...,6,4,2 (Fig. 3).
Next, in stage (ii), two segmented filters are constructed, using an analysis similar method to
[29], for the two sub-classes, 1
SPOFH and 2
SPOFH , respectively.
(iii) Selection and assignment optimized criterion used for segmentation of the Fourier plane
for both filters are realized as follows
, ,
,
, ,
, ,1
w ith 0, 1 ,
N ot affected otherw ise
l k
u v u vl
u v l k
i j i j
i j i j
SPO F
E ESpect if a
E E
Hl N
(4)
Here l
vuSpect
, is the spectrum of the reference Rl at position (u,v),
,
l
u vE is the energy of the
reference Rl at the position (u,v). The constant a, (here, set to 1.2), is a pixel assignment
coefficient. Basically, it means that the pixel of the filter is assigned to a given reference if and
only if its energy relative to spectral position (u,v) is larger than the energies of all other
10
references multiplied a times. Otherwise, it is not assigned. In Eq. (4), the usefulness of the
coefficient a is motivated by the fact that sometimes there exist areas of the spectral plane which
are very similar. Thus, by introducing this coefficient a these areas are not assigned to any
spectrum according to the segmentation criterion (Eq. 4). These areas will be assigned according
the neighborhoods of each spectrum (Fig. 4). A similar operation for the second filter 2
SPOFH is
applied using the spectra of the second sub-class and considering again Eq. (4).
(iv) In this procedure, unclassified pixels, i.e. those which do not satisfy Eq. (4), are
assigned to one of the two spectra by looking at their closest neighbors (Fig. 4) in order to avoid
isolated pixels. An isolated pixel represents a pixel of a spectrum l which is surrounded by pixels
of the spectrum k. Isolated pixels are detrimental to the segmented filter’s performance; the
effect being more and more important as the number of references which define the filter is
increased. Faced with several options for the purpose of decreasing this effect, we chose the
following scenario. The assignment of pixels which do not satisfy Eq. (4) is realized according
the schematic illustration depicted in Fig. 4. This figure show different situations, e.g. in Fig. 4
(a) and (b) the considered pixel is surrounded by pixels of only one spectrum.
Hence, the pixel is assigned to this spectrum. In another situation, Fig. 4 (c) illustrates the
case of a pixel surrounded by pixels belonging to the two spectra. In this case, the algorithm
searches for the immediate neighborhood ( 3 3 pixels) of the pixel, and the rule is to assign this
pixel to the spectrum which has the largest number of pixels. Fig. 4 (d) represents an example in
the case that the surrounding environment of the considered pixel contains exactly the same
number of pixels for the two spectra. In this case, the algorithm searches for the spectrum that
has the largest number of pixels in an enlarged neighbourhood, i.e. 5 5 pixels.
11
(v) Next, the assignment step is repeated. Interestingly, we observed that a single iteration is
necessary for letters.
(vi) Next, 1
SPOFH is multiplied with a symmetric filter (denoted as filter1(1/0) in Fig. 5, and
2
SPOFH is multiplied with the corresponding filter2(0/1) in Fig. 5. Finally, we sum the two filters
in the Fourier plane. Overall, the ASPOF can be described by
1 2
1 1 2 2ASPOF SPOF SPOFH p H filter p H filter , with
1p and
2p being two coefficients. In the
following we will study the effect of these two numbers on the performance of the ASPOF.
3.3. ASPOF parameterization
Two kinds of parameter were considered. On the one hand, to illustrate how t in Eq. (2), i.e.
2(2 1)t pixels are required for the calculation of the PCE, can impact the correlation
performances, we consider the letter A (with rotations between -90° and 90°) and compare the
performances of the POF, SPOF, AMPOF, and ASPOF. The fabrication of these filters was
realized using with 10 references, i.e. binary images of the letter A and 10 rotation angles. Fig. 6
compares the PCE resulting from correlating the letters A and V with the four composite filters.
Taking t=1 pixel (Fig. 6 (a)), the ASPOF is found to be very discriminating. One way to increase
the robustness of the ASPOF is to increase t. It can be seen that the ASPOF shows the best
compromise between robustness and discrimination factor when t=10. For t larger than 10, the
discrimination quality decreases (Fig. 6 (d)). In the following, t is set to 10. On the other hand,
we further show how p1 and p2 can influence the performances of the ASPOF. Fig. 7 (a) shows
the PCE values with p1=p2=1 where we choose the same illustrating example as above. We
found by an iterative method (100 iterations, Fig 7 (b)) that the best performances were obtained
by taking p1=2 and p2=1.5.
12
3.4. Composite filters used for comparison with the ASPOF
In this subsection, we will review various composite filters in order to select some of them
(POF, SPOF, AMPOF, and trade-off maximum average correlation height (OT-MACH)) that
will be compared with the ASPOF in order to demonstrate its robustness and its good
discrimination. We first study the effect of the background. We considered different classical
composite filters (the full list is shown in Table 1). A detailed comparative study of the
discrimination and robustness performances for standard composite correlation filters employed
for face recognition is presented in the supplementary material [30].
As expected the adapted composite filter is robust against rotation and not discriminating.
Moreover, its low discriminating character is more and more visible as the number of references
is increased. Our findings suggest that the composite POF is quite robust but not much
discriminating, see Fig. 8 and Fig. 9. For obtaining these figures we have correlated different
rotations (from -90 to 90 °) of the A and V letters with different composite POF filters using
different rotations of the letter A as reference images. As seen in Fig. 8 the different PCE data
show that the correlation values have an average PCE value larger than 3 10-3
. It is important to
point out that the PCE values obtained from the correlation of the letter V correlated with the
same filters have an average value less than 10-3
, and support the conclusions reported in [31].
In addition, composite POF show robustness to noise, especially when the noise is clearly
identified. However, we find that the energy contained in the correlation peak decreases
significantly, i.e. the PCE is decreased by a factor of 3 when using a POF containing 3 references
by contrast with a POF realized with a single reference (Fig. 8). This supports also the
conclusions made in [31]. Fig. (8) shows that for a 11-reference POF, the PCE is decreased by an
13
order of magnitude which renders unreliable the decision on the letter identification. Beyond 11
references, the weakness of the magnitude of the PCE makes the recognition of the images
forming the filter very difficult. This is partly due to the saturation. Indeed, when the number of
training images is large saturation occurs because the correlation filter is pixelated, i.e. since ach
pixel is encoded with a fixed number of bits, increasing the latter has for effect to slow down the
filtering process and thus to increase the required memory space. To overcome this saturation
limitation the segmented composite filter (SPOF) was proposed and validated in [8]. To further
show the interest in using a segmented filter with respect to the saturation problem which affects
the classical composite filter, we show in Fig. 2 (b) the 8-bit image of the sum (without
segmentation) of the three spectra corresponding to the reference images (Fig. 2 (a)). Fig. 2 (c)
shows the corresponding sum with segmentation [8]. Our calculations clearly indicate that the
image with segmentation shows significantly less saturation than that obtained without
segmentation.
For comparison, we performed simulations for a binarized composite POF [30]. Overall, the
behavior of this kind of filter is very similar to that of POF albeit with smaller performances. We
should notice that this conclusion is similar to that noted by Mohammed et al. in [32]. The low
performances of the binarized POF is confirmed by a comparison between the results obtained
by Mohammed et al. [32] and those reported by Rahman et al. [31].
Overall, we found that the composite POF has a good robustness against rotation but has a
low discrimination. To look at the discrimination of composite filters in more detail, we shall
consider the inverse composite filter. Such filter shows a strong discriminating ability and a low
robustness against small changes of the target image with respect to the reference image [30].
Fig. 10 presents the correlation results obtained for different rotations of the letter A and several
14
inverse filters. This leads to the conclusion that this inverse composite filter is not well adapted
for the composite architecture.
Next, testing the compromise optimal filter, we observed that its composite version is robust
to noise when the latter is clearly identified [30]. We also observed that the filter OT is not
robust to image rotation when the images are noisy, especially if the noise cannot be explicitly
evaluated, see Fig. 11 (different versions of this filter were considered in [30]).
The minimum average correlation energy (MACE) filter was also studied [11,33]. Its
correlation performance is good when the target image is similar to one of the reference images,
see the region B in Fig. 12. Nevertheless, the robustness of this filter is very poor since the
correlation peak disappears in the A and C regions (Fig. 12). In [34], Iftekharuddin et al.
proposed and validated an optimized MACE filter, the amplitude coupled MACE filter, that is
robust to rotation. However, as the classical MACE filter the optimized version is also sensitive
to noise, especially to structured noise.
Next, the AMPOF [16,35] was considered because it is high discriminating performance.
Awwal et al. [35-36] suggested an optimization of the POF filter based on the following idea: the
more the spectrum of the Fourier plane can be flattened the sharper will be the correlation peak.
These authors introduced the amplitude-modulated phase-only filter (HAMPOF). As shown in our
tests [30], the AMPOF is not robust to rotation and that is weakly robust to noise. However, the
AMPOF is found to have significantly superior correlation discrimination capability. In [37],
Iftekharuddin et al. studied a discretized version of the AMPOF filter. The resulting filter has
good recognition performances of a nosiy object. In addition, the good discrimination
performances of the AMPOF led Iftekharuddin et al. [38] to introduce an amplitude modulated
inverse filter (AMIF) which is robust to Gaussian white noise. But, the AMIF is very sensitive to
15
face rotation. Thus, we considered this AMPOF filter in order to compare it with our new
ASPOF filter. This is to highlight the good discriminatory performance of our ASPOF filter.
Another filter that was considered in this study is the OT-MACH filter [39-40]. From these
different comparisons, the POF, AMPOF, and the MACH composite filters were selected for the
comparison of correlation performances with our optimized ASPOF filter.
3.5. Performance criterion adapted to the ASPOF
Here, we will modify the definition of the PCE, i.e. Eq. (2), since we assumed that only a
single correlation peak is considered, and that the filter which led to the correlation peak contains
the total energy. These assumptions should be reconsidered for the ASPOF. Only half of the
energy of the output plane is associated with each filter since the ASPOF is composed of two
distinct filters. On the other hand, each filter leads to a correlation peak. Hence, for various
situations two correlation peaks can appear in the output plane (see, e.g. Fig. 13 (a)). To
circumvent the problem connected to the energy lowering of the correlation peak with the
ASPOF, i.e. only half the filter plane is used, the correlation peak magnitude could be multiplied
by a factor of 2. However, this “trick” is insufficient. In fact the asymmetry feature of this filter
leads to a correlation between the target image and a part of our filter, while there is
intercorrelation with the second part. This can be observed in Fig. 13 which shows three
correlation planes of the target image defined by the letter A.
As is apparent in Fig. 13, no correlation peaks are visible with the composite filters and
AMPOF. However, it should be noted that the correlation plane displayed in Fig. 13 (a) shows
two peaks: a first peak arises from the autocorrelation between the target image and the part of
the ASPOF which contains the reference image which is similar to the target image, and the
second peak can be identified as due to the intercorrelation between the target image and the
16
other part of the ASPOF. Overall, the autocorrelation plane is noised by the intercorrelation
plane. This effect has for effect to decrease the PCE values and consequently the identification
performances of the ASPOF. In addition, it may happen that the target image is similar to two
reference images, with one of them being in the first part of the ASPOF and the other one being
in the second part of the ASPOF. To address this difficulty, we defined a different criterion,
hereinafter denoted as the adapted peak-to-correlation energy (PCEA), which can be written as
0 0
0 0
0 0
0 0
2
2 2
1 1
2 ,
, 3 ,
x x t y y t
x x t y y t
x x t y y ty Mx N
x y x x t y y t
C x y
PC EA
C x y C x y
(5)
To further investigate this point, i.e. to separate the autocorrelation plane from the
intercorrelation one, we have added a specific phase term to the two parts of the ASPOF. This
phase is chosen that the correlation can be placed at a given point in the output plane. The
respective correlation plane performances of the different simulations are in very good
agreement with those gauged with the PCEA performance metric.
4. Illustrating the ASPOF performances
4.1 Robustness against face rotation
We considered the behavior of the 10-reference HASPOF for the problem of identifying the
letter A with a rotation angle ranging from -20° to 25°. Then, each letter A of the data-base
(obtained by rotating the A image over (-90°,90) in increments of 1° counter clockwise) was
correlated with the 10-reference HASPOF. As shown by the green line in Fig. 14 (a), the letter A
has been identified over (-20°,25°). To emphasize the benefit of the optimization stage
concerning the isolated pixels (stage (iv) of the algorithm) on the filter behavior, it is instructive
17
to contrast the results obtained for the HASPOF without taking into account this step in our
algorithm as shown by the green line in Fig. 14 (b). In the latter case, it should be noted that the
PCE values are decreased by a factor of 2. It is evident that this optimization stage allows us to
decrease the influence of the isolated pixels in the classical segmented filter HSPOF.
For comparison, we also plot in Figs. 14 (a) and (b) the correlation performances obtained
with 1
POFH (blue line), HSPOF (red line), and HAMPOF (black line) in the case of a noise free target
image. Figs. 14 (a) and (b) show that HASPOF is characterized by the largest PCE values in the
correlation region A, including those of the AMPOF. In addition, HASPOF presents the most
significant contrast between the correlation region A and the no-correlation regions B and C.
This shows that HASPOF is robust in the correlation region and discriminating in the no-correlation
regions.
1.1.1. Robustness to rotation and noise
As we next demonstrate, the conclusions are robust since ASPOF remains effective when
noise is contained in the target image. The noise analysis procedure was identical to that
performed previously. Fig. 14 (c), corresponding to a centred white noise of variance set to 0.1,
and Fig. 14 (d) which considers a structured background noise. For each case, HASPOF is a noise
robust filter by comparison with the composite filters which were considered previously.
Different tests involving other types of noise (not shown) are consistent with this observation.
4.2. ROC curves
The ROC curves in Fig. 15 demonstrate the effectiveness of the ASPOF for image
recognition. The data base consisted of 10 reference images of the letter A (binary image,
512 512 pixels) taking a range for angle of rotation from -60° to 60°, i.e. -55°, -45°, -30°, -15°,
18
0°, 15°, 30°, 45°, 55°, 60°. The correlation planes are quantified using Eq. (2) and Eq. (5) (we
choose t=10 pixels), and the recognition performance of composite filters for this specific set of
reference images are studied and compared (supplemental material [30]). The merging of the two
components of the ASPOF, i.e. 1 2
ASPOF SPOF SPOFH aH bH , was also optimized by choosing
a=2 and b=1.5. Fig. 15 shows the ROC curves obtained with the capital letters A and V (by
rotating the V letter in increments of 1° counter clockwise from -90° to 90°) for different
composite filters. As it can be seen in the ROC curves of Fig. 15 the AMPOF gives the closest
results to a random guess. This reflects the strong discrimination and the low robustness to
rotation of the AMPOF. From Fig. 15, we can see that the overall performance of the segmented
filter is better than that of the AMPOF, but remains poor because of the isolated pixels. POF is
able to achieve better performance than AMPOF; however due to the saturation problem only a
weak difference between the good decisions and the false alarms is observed (Fig. 14). This is in
contrast with the different performance points that can be achieved with the ASPOF. On one
hand, the results of our tests clearly show the good recognition performance of the ASPOF, i.e.
TPR=0.92 corresponding to FPR=0. On the other hand, it presents good separation between good
decisions and false alarms (Fig. 14).
4.3. Application to face recognition
The final point we wish to mention here is that the above ideas can be extended to the
problem of gray scale images identification. For this purpose we considered six different subjects
from the PHPID [2]; an example is given in Fig. 16 (a) with several training images per person
corresponding to different face orientations. As above, it is possible to define a 5-reference
ASPOF for a given subject (with variable rotation angles, i.e. from top to bottom: -15°, 0°, +15°,
19
and from left to right: +90°, 0°, -90) in the following manner: 3 references were placed in the
first half of the ASPOF and the remaining two references were positioned in the other half part
of the ASPOF.
In this example, we choose t=1 pixel, a=3 and b=1.5. The values of these parameters are
chosen in relation with the considered application. The 5-reference images corresponds to -45°,-
30°,-15°,15°,45° rotation angles (Fig. 16 (b)). ASPOF training set correlations in Fig. 17 (a) were
observed to produce good recognition performances in comparison to those of POF. It is likely
that these differences originate from the saturation problem which is more pronounced for the
case of faces. The recognition performance of the ASPOF was also compared to that of the
optimal trade-off MACH (OT MACH) filter with a background noise. The parameters for the OT
MACH are α=0,25, β=1, and γ=0,1 [30,39-40]. The results in Fig. 17 (b) show that the
recognition performances are significantly larger with the ASPOF. The result (Fig. 17 (b)) of our
method demonstrates the good recognition performance of the ASPOF, i.e. TPR=0.69
corresponding to FPR=0 to be compared with TPR=0.33 and TPR=0.47 for the OT MACH and
POF, respectively).
5. Summary
In summary, we have developed a face recognition system based on optical correlation for use
in identification and classification. The system will be able to detect, identify and track various
targets with robustness and discrimination. The method has two main stages: increase the
number of reference images in order to consider as much as possible target image changes, and
use composite filters. The main advantage of these filters is to merge the information of many
reference images in a single filter, thus reducing the number of correlations to make a reliable
20
decision. However, the robustness and discrimination performances decrease as the number of
reference images is increased, i.e. saturation. To overcome this limitations, we have presented
herein a fairly thorough analysis of an optimized composite filter, i.e. the asymmetric segmented
phase only filter (ASPOF), for improving the effectiveness of a VLC when used for face
identification. The principle of our filter consists in a specific segmentation of the filter’s Fourier
plane in several regions. Next, each region was assigned to a single reference image. This
optimization stage allowed us to increase significantly the number of reference images of the
segmented filter. However, the main drawback of this solution is to generate isolated pixels
which yield to low overall performances of this kind of filter. Next, we have proposed a new
type of optimized composite filter, i.e. the ASPOF. We found useful to introduce a new
segmentation criterion to characterize the correlation plane of this filter. Each of the stages has
various parameters that must be optimized in order to increase the robustness and discriminating
ability while increasing the number of reference images contained in the filter. We have
validated this technique and applied it to several binary and gray scale images identification.
Overall, our results are encouraging and demonstrate that the ASPOF is characterized by a good
compromise between robustness and discrimination.
Furthermore, we believe that the simplicity of the technique provides sufficient appeal
from the experimental viewpoint. Of equal or potentially more importance will be to optically
or numerically implement it. We believe that it would be interesting to explore this topic
further-we discuss very briefly below how this might be done. There are two key directions for
improving the results presented here. First, a careful optimization of the merging method is
required to maximize the use of the filter’s Fourier plane bandwidth. Second, we expect that
21
these general results will apply in 3D face recognition application as well as well. Work
addressing these avenues has been initiated.
Acknowledgments
The authors acknowledge the partial support of the Conseil Régional de Bretagne and thank A.
Arnold-Bos (Thales Underwater Systems) for helpful discussions. They also acknowledge S.
Quasmi for her help with the simulations. Lab-STICC is Unité Mixte de Recherche CNRS 3192.
22
References
1. A. VanderLugt, “Signal detection by complex spatial filtering,” IEEE Trans. Inf. Theor. 10,
139-145 (1964).
2. Pointing Head Pose Image (PHPID),
http://www.ecse.rpi.edu/~cvrl/database/other_Face_Databases.htm
3. C. S. Weaver and J. W. Goodman, “A technique for optically convolving two functions,”
Appl. Opt. 5, 1248–1249 (1966).
4. A. Alfalou and C. Brosseau “Understanding Correlation Techniques for Face Recognition:
From Basics to Applications,” in Face Recognition, Milos Oravec (Ed.), ISBN: 978-953-
307-060-5, In-Tech (2010).
5. J. L. Horner and P. D. Gianino, “Phase-only matched filtering,” Appl. Opt. 23, 812-816
(1984).
6. F. T. S. Yu, S. Jutamulia “Optical Pattern Recognition,” Cambridge University Press
(1998).
7. B. V. K. Vijaya Kumar, “Tutorial survey of composite filter designs for optical correlators,”
Appl. Opt. 31, 4773-4801 (1992).
8. A. Alfalou, G. Keryer, and J. L. de Bougrenet de la Tocnaye, "Optical implementation of
segmented composite filtering," Appl. Opt. 38, 6129-6135 (1999).
9. H. J. Caufield and W. T. Maloney, “Improved discrimination in optical character
recognition,” Appl. Opt. 8, 2354-2356 (1969).
10. C. F. Hester and D. Casasent, “Multivariant technique for multiclass pattern recognition,”
Appl. Opt. 19, 1758-1761 (1980).
23
11. A. Mahalanobis, B. V. K. Vijaya Kumar, and D. Casassent, “Minimum average correlation
energy filters,” Appl. Opt. 26, 3633-3640 (1987).
12. A. Alfalou and C. Brosseau, “Robust and discriminating method for face recognition based
on correlation technique and independent component analysis model,” Opt. Lett. 36, 645-
647 (2011).
13. M. Elbouz, A. Alfalou, and C. Brosseau, “Fuzzy logic and optical correlation-based face
recognition method for patient monitoring application in home video surveillance,” Opt.
Eng. 50, 067003(1)-067003(13) (2011).
14. I. Leonard, A. Arnold-Bos, and A. Alfalou, “Interest of correlation-based automatic target
recognition in underwater optical images: theoretical justification and first results,” Proc.
SPIE 7678, 76780O (2010).
15. V. H. Diaz-Ramirez, “Constrained composite filter for intraclass distortion invariant object
recognition , ” Opt. Lasers Eng. 48, 1153 (2010).
16. A. A. S. Awwal, “What can we learn from the shape of a correlation peak for position
estimation?,” Appl. Opt. 49, B40-B50 (2010).
17. A. Alsamman and M. S. Alam, “Comparative study of face recognition techniques that use
joint transform correlation and principal component analysis,” Appl. Opt. 44, 688-692
(2005).
18. S. Romdhani, J. Ho, T. Vetter, and D. J. Kriegman, “Face recognition using 3-D models:
pose and illumination,” in Proceedings of the IEEE 94, 1977-1999 (2006).
19. J. L. Tribillon, “ Corrélation Optique,” Edition Teknéa, Toulouse, (1999).
20. B. Javidi and C.- J. Kuo, “Joint transform image correlation using a binary spatial light
modulator at the Fourier plane,” Appl. Opt. 27, 663–665 (1988).
24
21. B. Javidi, “Nonlinear joint power spectrum based optical correlation,” Appl. Opt. 28, 2358–
2367 (1989).
22. M. S. Alam and M. A. Karim, “Fringe-adjusted joint transform correlation,” Appl. Opt. 32,
4344–4350 (1993).
23. M. S. Alam and S. F. Goh, “Three-dimensional fringe-adjusted joint transform correlation,”
Appl. Opt. 43, 3680–3685 (2004).
24. G. Keryer, J. L. de Bougrenet de la Tocnaye, and A. Alfalou, "Performance comparison of
ferroelectric liquid-crystal-technology-based coherent optical multichannel correlators,"
Appl. Opt. 36, 3043-3055 (1997).
25. A. Alfalou, “Implementation of Optical Multichannel Correlators: Application to Pattern
Recognition,” PhD Thesis, Université de Rennes 1–ENST Bretagne, Rennes-France (1999).
26. B. V. K. Vijaya Kumar and L. Hassebrook, “Performance measures for correlation filters,”
Appl. Opt. 29, 2997-3006 (1990).
27. J. L. Horner, "Metrics for assessing pattern-recognition performance," Appl. Opt. 31, 165-
166 (1992).
28. J. P. Egan, Signal Detection Theory and ROC Analysis (Academic Press Series in Cognition
and Perception, New York, 1975).
29. A. Pe'er, D. Wang, A. W. Lohmann, and A. A. Friesem, “Apochromatic optical correlation,”
Opt. Lett. 25, 776-778 (2000).
30. See supplemental material at http:// … for further technical details and data analysis
31. M. Rahman, A. A. S. Awwal and K. S. Gudmundsson, "Composite filter for search time
reduction for 3D model based object recognition", Proc. of SPIE 5201, 97-107 (2003).
25
32. T. Mohammad, M. Rahman, A. A. S. Awwal, and K. S. Gudmundsson, "Optical pattern
recognition of three dimensional images using composite binary phase only filters", Proc. of
SPIE 5556, 146-157 (2004).
33. D. Casasent and G. Ravichandran, “Advanced distorsion-invariant MACE filters,” Appl.
Opt. 31, 1109-1116 (1992).
34. K. M. Iftekharuddin, F. Ahmad, and M. A. Karim, "Rotation invariant target recognition
using amplitude coupled minimum average correlation energy filter", Opt. Eng. 35, 1009-
1014 (1996).
35. A. A. S. Awwal, M. A. Karim, and S.R. Jahan, “Improved correlation discrimination using
an amplitude-modulated phase-only filter,” Appl. Opt. 29, 233-236 (1990).
36. A. A. S. Awwal, W. A. McClay, W. S. Fergusson, J. V. Candy, J. T. Salmon, and P. J.
Wegner, "Detection and tracking of the back reflection of KDP images in the presence or
absence of a phase mask", Appl. Opt. 45, 3038-3048 (2006).
37. K. M. Iftekharuddin, M. A. Karim, P. W. Eloe, and A. A. S. Awwal, "Discretized amplitude
modulated phase only filter", Opt. Laser Technol. 28, 93-100 (1996).
38. K. M. Iftekharuddin, M. A. Karim, and A. A. S. Awwal, "Optimization of amplitude
modulated inverse filter", Math. Comput. Model. 24, 103-112 (1996).
39. A. Mahalanobis, B. K. V. Vijaya Kumar, S. Song, S. R. F. Sims, and J. F. Epperson,
“Unconstrained correlation filters,” Appl. Opt. 33, 3751–3759 (1994).
40. H. Zhou and T.- H. Chao, “MACH filter synthesizing for detecting targets in cluttered
environment for grayscale optical correlator,” Proc. SPIE 3715, 394 (1999).
26
Table
Table 1: Illustrating the different composite filters used in this study. Adcomp
H denotes the adapted
composite filter. It is realized by considering a linear combination of reference images and using
the adapted filter definition. HComp-POF is the POF composite filter. We tested two different
schemes for realizing the composite POF filter. In the first scheme (1
POFcompH ), a linear
combination of reference images was used to create the POF. The second scheme (2
POFcompH )
involves performing the POFcfor each reference, and using the linear combination of these
POFs. 1
BPOFcompH and
2
BPOFcompH are the binarized versions of the filters
1
POFcompH and
2
POFcompH ,
,respectively. The composite inverse filter IFcomp
H is the inverse filter of the linear combination of
reference images. The optimal composite filter OTcomp
H is realized by linearly combining
reference images. 1
SPOFcompH denotes the segmented filter realized by doing segmentation and
assignment with the energy criterion. The calculation of filter 2
SPOFcompH is done by replacing the
energy with the square of the real part of the different references spectra to be merged.
MACEcompH is the composite filter of the MACE filter.
AMPOFcompH is the composite version of the
AMPOF.
27
Figure Captions
FIG. 1: Schematic of the principle of the VLC scheme.
FIG. 2: Illustrating the saturation effect: (a) three 8-bit grey scale images. (b) Image obtained by
a classical linear combination of the three images shown in (a). (c) Image obtained using an
optimized merging (spectral segmentation).
FIG. 3: Technique used to classify the reference images in 2 sub-classes.
FIG. 4: Illustrating the optimized assignment procedure for isolated pixels.
FIG. 5: Merging technique based on the Fourier plane symmetry property.
FIG. 6: (Color online) PCEs obtained with 10-reference POF, SPOF, AMPOF, and ASPOF. (a)
t=1 , (b) t=5, (c) t=10, and (d) t=20.
FIG. 7: (Color online) PCEs obtained with a 10-reference ASPOF and t=10 pixels. (a) p1=p2=1
(b) p1=2 and p2=1.5.
FIG. 8: (Color online) PCEs obtained with the composite POF. The colors shown in the inset
denote the different filters as a function of the number of references used.
28
FIG. 9: (Color online) Discrimination results: (a) the target image is the letter V. (b) PCEs
obtained with a filter fabricated with reference images of letter A. The colors shown in the inset
denote the different filters as a function of the number of references used.
FIG. 10: (Color online) PCEs obtained with the inverse composite filter. The colors shown in the
inset denote the different filters against the number of references used.
FIG. 11: (Color online) (a) Illustrating the letter A with additive background noise. (b) Same as
in (a) with a rotation angle of -50°. (c) Illustrating the letter A with structured noise. (d) Same as
in (c) with a rotation angle of 50°. (e) Illustrating the letter A for a weak contrast. (f) PCEs
obtained with the OT composite filter taking α=0.6. The colors shown in the inset denote the
different filters versus the number of references used.
FIG. 12: (Color online) PCEs obtained with a 10-reference MACE when the target images are
noiseless. Several examples of the rotated letter A are shown at the bottom of this figure. The
inset shows two correlation planes. On the right hand: autocorrelation obtained without rotation,.
On the left hand, inter-correlation obtained with the letter A oriented at -75°.
FIG. 13: Output normalized Fourier planes obtained by correlating of the same target image
(letter A with a rotation angle of -75°) with: (a) ASPOF, (b) composite filter, and (c) AMPOF.
FIG. 14: (Color online) Comparison between the different correlations of letter A (we consider
rotation angles ranging between -90° and 90°) with the 10-reference composite filters: POF (blue
29
line), Segmented (red line), AMPOF (black line), and ASPOF (green line). (a) PCEs obtained
using the optimization stage concerning the isolated pixels, (b) PCEs obtained without the
optimization stage concerning the isolated pixels. (c) and (d) represent the PCEs obtained with
noised target images.
FIG. 15: (Color online) ROC curves obtained with 10-reference composite filters: POF (red),
SPOF (green), AMPOF (purple) and ASPOF (navy blue). The sky-blue line shows the random
guess.
FIG. 16: Example of face images from PHPID, that were captured under variable angle
conditions, i.e. from top to bottom: +15°, 0°, +15°, and from left to right: -90°, 0°, +90°: (a)
target faces, (b) references.
FIG. 17: (Color online) (a) ROC curves obtained by correlating faces of a given subject, e. g.
Fig. 9 (a), with 6 other individuals with 5-reference ASPOF (navy blue) and POF (red)
composite filters. The sky-blue line shows the random guess. (b) ROC curve obtained with an
OT MACH.
30
Composite filter Notation
Adapted filter Adcomp
H
Phase-only filter 1
POFcompH ,
2
POFcompH
Binary phase-
only filter
1
BPOFcompH ,
2
BPOFcompH
Inverse filter IFcomp
H
Compromise
optimal filter OTcomp
H
Segmented filter 1
SPOFH ,
2
SPOFH
Segmented binary
filter
1
BSPOFH , 2
BSPOFH
Minimum
average
correlation
energy filter
MACEcompH
Amplitude
modulated phase-
only filter
AMPOFcompH
Table. 1: Leonard, Alfalou and Brosseau
31
Imagespectrum
TargetImage
FTX
FT -1
Correlation
Plane
FTReferenceImage
Referencespectrum
Filter
creation
FIG. 1: Leonard, Alfalou and Brosseau
32
FIG. 2: Leonard, Alfalou and Brosseau
33
FIG. 3: Leonard, Alfalou and Brosseau
34
?? ?
? ? ? ?
?
?
S = Spectre l
S = Spectre k
(a) (b) (c) (d)
Isolated pixel not assigned:
before
Isolated pixel assigned:
after
k
l
FIG. 4: Leonard, Alfalou and Brosseau
35
FIG. 5: Leonard, Alfalou and Brosseau
36
FIG. 6: Leonard, Alfalou and Brosseau
37
FIG. 7: Leonard, Alfalou and Brosseau
38
FIG. 8: Leonard, Alfalou and Brosseau
39
(a)
(b)
FIG. 9: Leonard, Alfalou and Brosseau
40
FIG. 10: Leonard, Alfalou and Brosseau
41
(a)
(b)
(c)
(d)
(e)
(f)
FIG. 11: Leonard, Alfalou and Brosseau
42
FIG. 12: Leonard, Alfalou and Brosseau
43
IntercorrelationAutocorrelation
ASPOF (a)
Composite (b)
AMPOF (c)
0
0.5
1
0
0.5
1
0
0.5
1
0
200 400 600
Image size
0 200 400 600
Image size
0 200 400 600
Image size
FIG. 13: Leonard, Alfalou and Brosseau
44
FIG. 14: Leonard, Alfalou and Brosseau
45
FIG. 15: Leonard, Alfalou and Brosseau
46
FIG. 16: Leonard, Alfalou and Brosseau
47
FIG. 17: Leonard, Alfalou and Brosseau