a method of creating 3-d face images from 2-d photos for face recognition
TRANSCRIPT
-
8/2/2019 A Method of Creating 3-D Face Images From 2-D Photos for Face Recognition
1/16
-
8/2/2019 A Method of Creating 3-D Face Images From 2-D Photos for Face Recognition
2/16
A method of creating 3-D face images from 2-D photos 41
1 Introduction
Biometrics, the technology of performing personal identification or authentication via an
individuals physical attributes, is becoming an increasingly viable solution for identity
management, information protection and homeland security in our modern society.
Biometrics, including fingerprint, iris, face, voice and DNA, provide an additional layer
of protection when used along with other security solutions. For applications requiring
higher levels of security, biometrics can be integrated with other authentication means
such as smart cards and passwords to reduce both false acceptance rate and false rejection
rate (Nanavati et al., 2002).
In this paper, we focus mainly on issues related to 3-D face recognition. Statistically,
every face has special features that define a particular individual, and, meanwhile, faces
can also be very similar in a particular feature space. This makes face recognition an
interesting and difficult problem. Face recognition with high speed and robust accuracyhas been a specialised application area in the field of computer vision for many years.
Numerous theoretical and implementational advances have made automatic-face-based
authentication not only technically feasible, but also economically practical.
In a 2-D face recognition system, whenever its illumination condition or pose angle
changes, the performance of state-of-the-art systems can be greatly decreased (Blackburn
et al., 2001). Therefore, for robust face recognition, it is necessary to utilise a face model
to compensate for the variation of pose and illumination differences prior to feature
extraction. Furthermore, 2-D images have only intensity information and are
lack of depth information. These problems can be fixed with 3-D approaches.
To effectively conduct pattern analysis, pre-processing of 3-D face images is essential.
Early work included developing algorithms that matched a small number of feature
vertices to image positions and interpolated deformations of the surface in between
(Huang and Tang, 1996) using deformation models (Lowe, 1991), or analysed images
with shape-from-shading (Zhao and Chellappa, 2000). An excellent work on 3-D face
recognition was done by Blanz and Vetter (2003), Blanz et al. (2005, 2007) and
Blanz (2006). In Blanz and Vetter (2003), deformable 3-D models were combined with a
computer graphics simulation of projection and illumination. Given a single image of a
person, the algorithm estimated 3-D depth, intensity and all relevant 3-D scene
parameters. In Blanz (2006), two algorithms for 3-D face recognition and reconstruction
were described, one reproducing the full appearance for the face while the other based on
a set of feature point locations.
In our approach, a 3-D parametric face database is first developed, which is
represented by 2-D intensity (or texture, similar to a black and white photo) files and files
containing 3-D coordinates of the face at sampling points. When a 2-D probing photo is
presented to the system, a 3-D synthetic face image, which starts with a neutral 3-D facemodel derived from the database, is projected onto the 2-D image plane, following pose
rotation and illumination compensation procedures. The purpose of these procedures is to
make sure that the projected 2-D synthetic face image has the same pose angle and
illumination as the 2-D probing photo. An optimisation algorithm is applied to minimise
an error defined in terms of the difference between the parameters of the projected 2-D
synthetic image and those of the 2-D photo by modifying the 3-D depth and intensity
parameters of the synthetic face model. Once the algorithm converges, these depth and
intensity parameters are the best representation of the 2-D probing photo in the 3-D
-
8/2/2019 A Method of Creating 3-D Face Images From 2-D Photos for Face Recognition
3/16
42 X. Guan and H. Zhuang
space. And, this 3-D parametric representation of the probing image is used for
recognition by comparing with those stored in the gallery database.One advantage of this method is that only the depth and intensity parameters of
gallery images need to be stored in the database for biometric authentication purpose,
which can significantly reduce the size of the database. The second one is that only
2-D photos are used for probing, which makes the implementation very practical since no
real-time 3-D face photo is required in the verification stage. In comparison with the
method provided in Blanz and Vetter (2003), an advantage is that potentially one does
not need to register any landmark points, which makes it a complete autonomous
procedure. Additionally, our method can be easily modified to handle more than one
probing photos from the same person, the more probing we have, the better recognition it
will lead to. This method provides a better recognition rate when compared with our
previous method, since the recognition is implemented in the 3-D space instead of the
2-D plane, and both the 3-D depth and the intensity information are provided forrecognition rather than only the 2-D intensity information (Guan and Zhuang, 2007).
The results from an experimental study presented in the paper illustrate the effectiveness
and robustness of the proposed approach.
The remainder of the paper consists of the following sections. Section 2 outlines the
solution method, Section 3 provides the implementation details, and Section 4 presents
the results of experimental studies. The paper ends with concluding remarks.
2 Solution method
2.1 Solution strategy
A human face may be modelled by a combination of faces. To illustrate the concept, letus think about a child who has some facial features resembling her father, mother, sister,
brother, grandmother or grandfather (Figure 1). Furthermore, certain features of a face
may be similar to faces of other unrelated individuals.
In this research, it is assumed that a 3-D face image is decomposed into a depth
feature vector z and intensity feature vector t. A face model (zf, tf) may then be
represented as a linear combination of depth and intensity vectors shown in equation (1),
wherei
fz and
f
it , i = 1, , k, are the ith depth and intensity vectors, respectively, ai and
bi are the weights (parameters) representing the contribution of the ith vector, and kis the
number of vectors. The task is now to select ai and bi to make a new face (zf, t
f) similar
to the probing face.
Figure 1 Conceptual illustration of linear combination of faces (see online version for colours)
1 1
a , bi
k kf f f f
i i i
i i= =
= = z z t t (1)
-
8/2/2019 A Method of Creating 3-D Face Images From 2-D Photos for Face Recognition
4/16
A method of creating 3-D face images from 2-D photos 43
Since the main application of the proposed method is biometric authentication, more
discussion on this subject is given next. This automatic authentication system is made upof three stages, model database creation, enrolment and verification. In the model
database creation stage, a 3-D face database is first developed, utilising a number of 3-D
model face images. In the enrolment stage, individuals who are supposed to have the
authority of accessing are registered with their 2-D photos. Then, a 3-D synthetic face
derived from these 2-D photos for each individual subject is created and represented by
parameters of 3-D model faces. In the verification stage, only 2-D face images are
captured and compared with the 3-D images stored in the gallery database for
authentication. A 3-D face model is created by selecting weights of 3-D depth and
intensity such that the projected 2-D image of the 3-D face is similar to the 2-D probing
image. Only the 3-D weights are needed to be saved in the database, then the general
feature extraction procedure is applied to compare the probing images 3-D parameters
with those saved in the database to determine the identity of the person. A detaileddescription of these stages is given next.
2.2 Model database creation stage
Each 3-D model face image is composed of a depth image and an intensity image.
Following the pre-processing, a 3-D feature extraction procedure, such as Principal
Component Analysis (PCA) or Fisher Linear Discriminant (FLD), is then performed on
both the depth and the intensity images in the database, respectively. After this, 3-D face
parameter vectors, consisting of the depth and intensity information of 3-D images of the
subjects, are obtained, which will in turn be saved in a database.
2.3 Enrolment stageTwo-Dimensional face images of subjects to be enrolled are first taken in this stage.
The 2-D gallery images are converted into 3-D synthetic face by utilising the 3-D model
we created in the previous stage, and only the depth and intensity parameter vectors
are stored to represent each gallery image.
2.4 Verification stage
The proposed scheme for the verification stage is shown in Figure 2. The input to the
procedure is a 2-D probing face photo. Initially, a 3-D neutral face is created from the
average 3-D face derived from the gallery database. A pose angle determination
procedure is applied to determine the viewing angle of the 2-D probing face
approximately, and the initial 3-D face is then rotated to align with the probing face.Afterwards, an illumination compensation algorithm is then applied to the 3-D face so
that its 2-D projection has the similar intensity profile as the 2-D probing image. The 3-D
synthetic face image is then projected to the 2-D image plane to compare with the
probing photo in the 2-D parametric space. The discrepancy between the two images will
lead to another iteration in which a new set of 3-D parameters are generated in an effort
to reduce the discrepancy. The iterative process continues until the algorithm converges
or the number of iterations has exceeded a preset value. These sets of parameters are then
used to represent the 3-D face model, which resembles the subject in the 2-D probing
photo.
-
8/2/2019 A Method of Creating 3-D Face Images From 2-D Photos for Face Recognition
5/16
44 X. Guan and H. Zhuang
Figure 2 Verification stage
As mentioned earlier, any feature extraction algorithm can be used at this point to
compare the 3-D depth and intensity parameters of the probing photo and the gallery
images. In our research, both FLD and PCA methods are used to perform face
recognition with a very high recognition rate.
3 Major algorithms
The algorithms for initial pose determination, illumination compensation andoptimisation are somewhat standard procedures. For the readability of the paper, they are
provided in this section. Readers who are familiar with these algorithms may skip this
section.
3.1 Initial pose determination
Our starting point in this study is to assume that as the face changes its pose, the
corresponding change in the 2-D positions of the projected features can be approximated
by an affine transformation. The estimation of the pose angle of the 2-D probe photo uses
-
8/2/2019 A Method of Creating 3-D Face Images From 2-D Photos for Face Recognition
6/16
A method of creating 3-D face images from 2-D photos 45
the affine transformation between the feature positions in the 2-D probe photo and their
corresponding positions in the 3-D synthetic face image in which the view of the face isfronto-parallel (Yao et al., 2001). In our study, the four feature points we use are the
centre of each eye, the middle of the month and the nose tip.
The three rotation angles, yaw, pitch and roll, are described in Figure 3.
In this study, we ignored pitch angle and focused on yaw and roll angles, as the pitch
angle is very small for the faces saved in our database. The framework, though, is still
valid when the pitch angle cannot be ignored, in which case a more sophisticated
algorithm (Ullman and Hutternlocher, 1990) needs to be adopted for pose determination.
Figure 3 The yaw, pitch and roll angle (see online version for colours)
Let (u, v) be the 2-D image coordinates and (x,y,z) be 3-D coordinates. The resulting
3-D synthetic image needs to be rotated so that its 2-D projection matches the probing
photo. Under the weak perspective transformation assumption, (u, v) can be related to
(x,y,z) as follows:
0 0 0( ) ( , , ) ( , , )
1
x
u ys x y z
v z
=
S R T (2)
where S(s) is the scaling matrix, R(, , ) is the rotation matrix in roll, pitch and yawangles, respectively, and T(x0,y0,Z0) is the translation matrix (Fu et al., 1987).
To find the solution for the unknowns, Huttenlocher and Ullman proposed a
closed-form solution under the weak perspective transformation assumption (Ullman and
Hutternlocher, 1990), where 3 points in both the 2-D image plane and the 3-D space are
used for face alignment. However, the drawback of this method is that the error will
propagate, resulting in large errors to the variables solved in the later steps.
We implement an alternative iterative method also under the weak perspective
transformation assumption. To obtain an initial condition of the iterative algorithm, we
-
8/2/2019 A Method of Creating 3-D Face Images From 2-D Photos for Face Recognition
7/16
46 X. Guan and H. Zhuang
assume the pitch angle is relatively small, and the roll angle can be roughly compensated
in the 2-D image plane. Therefore, equation (2) is simplified to
0
0
0
cos 0 sin 0 1 0 0
0 0 0 0 0 0 0 0 1 0.
0 0 0 sin 0 cos 0 0 0 1
0 0 0 1 0 0 0 1 1
x x
u s y y
v s z z
=
(3)
The translation part T(x0,y0,z0) can be removed:
1 2 1 2 1 2(cos ( ) sin ( ))u u s x x z z = + (4)
1 2 1 2( ).v v s y y = (5)
From equation (5),s can be solved, and from equation (4), the yaw angle can be solved.Then,x0 andy0 can be determined. Note that under the weak perspective transformation,
z0 is not relevant. Once the initial condition is obtained, a non-linear least-squares
algorithm can then be applied to find all the unknown parameters more accurately.
3.2 Illumination compensation
The Phong reflection model (Watt, 2000) is a shading model used heavily in 3-D
computer graphics for assigning shades to each individual pixel of an object. It was
developed by Bui Tuong Phong in 1973. It considers the reflection from a surface
to consist of three linearly combined components, ambient, diffused and specular.
The ambient component is a constant one that simulates global or indirect illumination.
This component is necessary because parts of a surface that cannot see the light source,
but can be seen by the viewer, need to be lit. Otherwise, they would be rendered as black.
In reality, such lighting comes from global or indirect illumination.
It is useful to consider what type of surface such a model simulates. Linear
combination of a diffuse and specular component occurs in polished surfaces, and
specular reflection results from the transparent layer and diffuse reflection from the
underlying surface (Ishiyama et al., 2002).
Suppose the illumination is in the direction l, with irradianceI() =I(, I). Let n be
the surface normal, and m be the mirror reflection direction (Figure 4). Then, the
reflected radiance used by the Phong reflectance model at a surface pointxp, per unit area
perpendicular to the viewing direction v, is
Lights
( , , ) ( ) ( ( )( ) ( ) ( )( ) )ek
p a d sR x v k r k r n I k S v = + + l m (6)
where r() is the diffuse spectral reflectance distribution for the surface, ka, kd and ks arenon-negative coefficients for the ambient, diffuse and specular reflection terms,
respectively. In our experiment, ka = 0.1, kd= 0.05 and ks = 0.05. ke is the spectral
exponent, controlling the spread of the specular reflection (rougher surfaces modelled by
smallerke and in our experiment 5.0 was used); S() is the spectral distribution of thespecular reflection. It is just I() for painted or plastic surfaces. For metals, it can beapproximated by some linear combination ofI() and r().
-
8/2/2019 A Method of Creating 3-D Face Images From 2-D Photos for Face Recognition
8/16
A method of creating 3-D face images from 2-D photos 47
Figure 4 (a) 3-D face shape with (b) Phong reflectance model
(a) (b)
The basic way of rendering a scene is on a polygon-by-polygon basis, where each
polygon is rendered one by one independently. The common hidden surface removal
algorithm that is compatible with this method is Z-buffer (1975). The combination
of the Z-buffer algorithm, the Phong model and interpolator represents one of the most
popular rendering options (Wand et al., 2001). Pixels in the interior of a polygon
are shaded, using an incremental shading scheme, and their depth is evaluated
by interpolating thezvalues of the polygon vertices after a viewing transformation has
been applied. This algorithm searches interior polygon points to find for each point (x,y)
its minimum z value. This search is usually implemented by using a Z-buffer, which
holds for (x, y) the smallest z value so far encountered. During the processing of a
polygon, one decides whether to write the intensity of (x,y) into the frame buffer or not,
depending on whether the depthzof (x,y), is less than the depth so far encountered in the
Z-buffer.
3.3 Optimisation method
To recreate a 3-D face, we need to estimate the depth and intensity parameters. For this
purpose, a gradient-based optimisation procedure is employed, which includes the
formulation of a cost function, initial conditions, an update rule and terminal conditions.
The optimisation problem is to choose the parameter vectorsuch that the following costJis minimised:
( ( )) ( ( ))TJ g g g g = (7)
where g is the 2-D intensity vector, and ( )g is the intensity vector of the 2-D image projected from a 3-D synthetic image, which is represented in its parametric form .
If more than one 2-D probing photo are provided, the 3-D synthetic face image needsto be rotated and illumination compensated multiple times to match each of the probing
photos, and vectorsg and ( )g also need to be extended to include all the intensity
vectors.
The proposed update rule is shown in equations (8) and (9),
1 j j j j + = + (8)
( ( ))j jf g g = (9)
-
8/2/2019 A Method of Creating 3-D Face Images From 2-D Photos for Face Recognition
9/16
48 X. Guan and H. Zhuang
where the subscriptj indicates thejth iteration, j is the parameter vector, aj is a number
between 0 and 1, controlling the size of the parameter adjustment, j is the parameteradjustment produced by equation (9), and f(.) is a function that calculates the parameter
adjustment based on the errors between the given and the estimated feature vectors. For
the non-linear least-squares problem, an often-used practice is to choose f as a linear
mapping realised with Singular Value Decomposition (SVD).
To start the iteration, the average face of the models in the database is mapped to
the parameter space, from which an initial condition for the parameter vector isobtained. The algorithm can be terminated when either the cost function is less than a
prescribed threshold value, or the number of iterations exceeds a prescribed number
(Levenberg, 1944; Marquardt, 1963).
3.4 Feature extraction algorithms
Feature extraction algorithms include PCA and FLD. PCA is a technique mainly for
removing redundancies in a given data set. It has been also applied extensively for both
face representation and recognition (Turk and Pentland, 1991; Swets and Weng, 1996;
Belhumeur et al., 1997; Etemad and Chellappa, 1997). However, it has to be mentioned
that the PCA method only encodes covariance of face images, and such second-order
statistics provide only partial information of the given images. In this research, we used
the 3-D PCA method to extract both the depth and the intensity features for face
recognition. Future works include replacing PCA with more advanced feature extraction
methods to incorporate higher-order statistics (Wiskott et al., 1997; Liu and Wechsler,
2000).
In the PCA method, it attempts to maximise the scatter of the training images in face
space. On the other hand, the FLD method attempts to maximise the between-class
scatter, while minimising the within-class scatter. In other words, it attempts to move
features of the same face closer together, while moving features of difference faces
further apart (Belhumeur et al., 1997).
Since both the PCA and the FLD algorithms are popular methods, we refer the reader
to references such as Turk and Pentland (1991), SAS Publishing (2007) and Xiong et al.
(2005) for details.
4 Experimental study
4.1 System set-up
The 3-D camera system, developed by Genex Technologies, Inc., consists of a 3-Dfacecam camera that takes 3-D pictures, a digital camera that takes 2-D photos, a 3-D
computer platform with integrated frame grabber board, a video card and capture
software that controls the operation of the 3-D camera and also allows the user to view
and edit 3-D pictures.
The 3-D camera system was a 3-D surface profile measurement system capable
of acquiring full-frame dynamic 3-D images of objects with complex surface geometry.
The (x, y, z) coordinates for all visible points on the object surface were provided by a
single 3-D image. The three-sensor system captured over 300,000 data points of
-
8/2/2019 A Method of Creating 3-D Face Images From 2-D Photos for Face Recognition
10/16
A method of creating 3-D face images from 2-D photos 49
geometric and true colour intensity information. A snapshot of the camera system at our
laboratory is given in Figure 5.
Figure 5 A snapshot of the 3-D camera system in the signal-processing lab(see online version for colours)
The computational cost to seek an optimal solution for removing the discrepancy between the 2-D probing photo and the 3-D synthetic model face was quite high.
In our lab, we used Intel Core 2 Duo CPU T7300 @ 2.00 GHz, with 2.00 GB of RAM.
The average case was about 5 min for a probing image. To manage fairly reasonable
experiments, we created a 3-D face database from 46 persons with different age, sex and
race. For each subject, 2 out of 9 3-D pictures taken from various viewing angles under
different lighting conditions were enrolled to create the gallery database, and another 2-D
digital picture of each subject was used as the test image. These 3-D and 2-D pictures
were the basis for our experiments.
In the first experiment, we implemented the scheme presented in Sections 2 and 3
with the PCA feature extraction procedure. In the second experiment, the FLD procedure
was applied to check if different feature extraction procedures would produce different
recognition rates.
4.2 Pre-processing
Pre-processing consists of the following five steps in converting a Genex GTI 3-D image
to a normalised image:
GTI format to pseudo-stereo-lithography conversion: Convert the raw data into the
pseudo-stereo-lithography format, which is a list of triangular surfaces that describe
both the depth and the intensity information for each vertex.
-
8/2/2019 A Method of Creating 3-D Face Images From 2-D Photos for Face Recognition
11/16
50 X. Guan and H. Zhuang
Geometric normalisation: Use both eye coordinates to align the yaw and roll angle.
Specifically, thexy coordinates are used to align the roll angle and the depth ofboth eye are used to align the yaw angle, so that all the 3-D images rotate to the
fronto-parallel view and both eyes are in the same horizontal line.
Generating 2-D depth and intensity matrices: Apply 3-point interpolation to the
pseudo-stereo-lithography file to generate separate depth matrix and intensity matrix.
In the future work, we will use 6-point interpolation to fit the curved surface better.
Masking: Crop the 2-D depth and intensity matrices using an elliptical mask and
image borders such that only the face from forehead to chin and cheek to cheek is
visible.
Histogram equalisation: Equalise the histogram of the unmasked part of the image.
Figure 6 shows an example of the depth image, and the corresponding intensity image.
Figure 6 (a) Example of 2-D depth image and (b) example of 2-D intensity image
(a) (b)
4.3 Initial pose determination
Figure 7(a) is the 2-D fronto-parallel projection of a 3-D intensity image, and Figure 7(b)
is a probing 2-D image. We applied the pose determination algorithm outlined in
Section 3.1 to estimate the pose of the 2-D probing image. We then rotated the
3-D image to the computed pose angle and projected it to the 2-D image plane.
The resulting 2-D image is shown in Figure 7(c).
Figure 7 Sample image of rotation: (a) the frontal view from a 3-D image; (b) a 2-D probe imageand (c) the 3-D image after pose rotation
-
8/2/2019 A Method of Creating 3-D Face Images From 2-D Photos for Face Recognition
12/16
A method of creating 3-D face images from 2-D photos 51
4.4 Illumination compensation
The Phong reflection model and the Z-buffer rendering method described in Section 3.2
were used in this experiment. And, both the reflection coefficient and the light source
location are the variables that we try to optimise.
Figure 8(a) shows a 2-D test image, and Figure 8(b) and (c) shows the recreated
images after the pose compensation, among which Figure 8(b) shows the one without any
illumination compensation, and Figure 8(c) shows the compensated one utilising 3 lights
with the Phong reflection model.
Figure 8 Illumination compensation example: (a) probe image; (b) no compensation;(c) compensation with 3 light sources
4.5 Face recognition experiment results
Images of 46 subjects were stored in our 3-D gallery face database, and each subjects
frontal view 3-D image and one slightly angle view image were used, so the gallery
database includes the total of 92 images. Additionally, a third 2-D picture with different pose angle view was also taken and used as probing photos. Two face recognition
experiments were conducted: In the first experiment, we implement the PCA method,
while in the second experiment, we use the FLD method. Both PCA and FLD have been
applied extensively for face representation and recognition (Turk and Pentland, 1991;
Swets and Weng, 1996; Belhumeur et al., 1997; Etemad and Chellappa, 1997). However,
it has to be mentioned that these methods only encode covariance of face images, and the
second-order statistics provide only partial information of the given images.
4.5.1 Experiment using PCA for feature extraction
In this experiment, we first positioned each 3-D gallery image to the same pose as the
2-D probing photo, and then applied the Phong reflection model with the optimisation
algorithm outlined in Section 3.2 for light compensation. This procedure was iterativelysolved until the algorithm converges. The depth and intensity parameters of the resulting
3-D synthetic image were then used for face recognition against each of the 3-D gallery
images.
Figure 9 shows an example of the test result. Figure 9(a) and (c) is the same 3-D
synthetic image generated to match Figure 9(d), the 2-D probing photo. And, in the
recognition part, Figure 9(a) is used to compare with Figure 9(b), the 3-D gallery images
stored in the database.
-
8/2/2019 A Method of Creating 3-D Face Images From 2-D Photos for Face Recognition
13/16
52 X. Guan and H. Zhuang
Figure 9 (a) The 3-D synthetic image in front view; (b) the 3-D gallery image in front view;
(c) the same synthetic image in semi-profile view and (d) the 2-D probing imagein semi-profile view
Figure 10 shows the errors vectors between the feature vectors of the probing photos
and the projected synthetic images. For the recognised faces, the mean distance was
2.0939e+006 and the standard deviation was 2.6251e+006; for other faces, the mean
distance was 1.4479e+007 and the standard deviation was 1.5372e+007.
Figure 10 Error norms between each of the probe and gallery images: The horizontal axis denotesthe image index, and the vertical axis denotes the base-10 logarithm of error norm interms of the PCA feature vectors (see online version for colours)
The above-mentioned experiment used 10 most significant eigenvectors in PCA for both
intensity and depth data. Comparison tests show that when the size of eigenvectors is
doubled, the recognition rate was only improved by 2%, while the calculation cost
increased significantly. On the other hand, reducing the size to eight significantly
decreased the recognition rate to 84.8%. The results from this comparison study are given
in Table 1.
-
8/2/2019 A Method of Creating 3-D Face Images From 2-D Photos for Face Recognition
14/16
A method of creating 3-D face images from 2-D photos 53
Table 1 Recognition rate comparison
Recognition rate (%) False positive rate (%)
20 Eigen vectors 93.5 6.5
10 Eigen vectors 91.3 8.7
8 Eigen vectors 84.8 25.2
4.5.2 Experiment using FLD for feature extraction
In this experiment, we implement the FLD method under the same conditions. Figure 11
shows again the error vectors between the feature vectors of the probe photos and the
projected gallery images. For the recognised faces, the mean distance was 1.052e+6 and
the standard deviation was 1.278e+006; for the other faces, the mean distance was
1.081e+007 and the standard deviation was 1.178e+007. And, the recognition rate was91.3% as well.
In comparison with the result from the first experiment, one observes that with FLD,
the difference of the average errors between the feature vectors of the recognised face
and the unrecognised ones was larger, and the standard deviation of errors within each
group was smaller as well. This means that within both the accepted and the rejected face
groups, the members were clustered tighter, and the two groups were more separated,
which led to a better recognition.
Figure 11 Error norms between each of the probe and gallery images: The horizontal axis denotesthe image index, and the vertical axis denotes the base-10 logarithm of error norm interms of the FLD feature vectors (see online version for colours)
We also compared the result of intensity parameters only with combined intensity and
depth parameters, with the weight of 0.1 for depth vectors. The results from thiscomparison are displayed in Table 2. The False Positive Rate here describes the rate of
matching a probe photo to a wrong gallery image.
Table 2 Recognition rate comparison
Recognition rate (%) False positive rate (%)
Intensity parameters only 89.1 10.9
Intensity and depth parameters 91.3 8.7
-
8/2/2019 A Method of Creating 3-D Face Images From 2-D Photos for Face Recognition
15/16
54 X. Guan and H. Zhuang
4.5.3 Experiment on AT&T 2-D database
We applied the 3-D face modelling process on the 2-D AT&T face database, with the
total of 200 gallery 2-D images, to gather the 3-D depth and intensity parameters.
Some of recreated 3-D faces are shown in Figure 12. In the figure, the first row depicts
the input images of the subjects, which is the 2-D face photo. The second row is the
recreated 3D images aligned with the input photos.
Figure 12 Recreated 3-D faces from 2-D images (see online version for colours)
5 Conclusions
A method for 3-D face recognition by creating 3-D face images from single 2-D photo
using 3-D models has been proposed in this paper. A major difference between the
proposed method and those in the literature is that in the new method, the discrepancy
between the feature vectors of a 2-D probing photo and a projected gallery image is used
to guide an optimisation algorithm, which seeks an optimal solution in a 3-D face
parametric space. This is a simple yet effective procedure, which is demonstrated by the
experimental studies. The method has applications to biometric authentication and 3-D
graphics. Future studies include adding more 2-D gallery and probing images as well as
applying the method for 3-D face recreation.
References
Belhumeur, P.N., Hespanha, J.P. and Kriegman, D.J. (1997) Eigenfaces vs. Fisherfaces:recognition using class specific linear projection, IEEE Trans. Pattern Analysis & MachineIntelligence, Vol. 19, No. 7, pp.711720.
Blackburn, D.M., Bone, M. and Phillips, P. (2001) FRVT 2000: Facial Recognition Vendor Test,Evaluation Report, available at: http://www.frvt.org/DLs/FRVT_2000.pdf
Blanz, V. and Vetter, T. (2003) Face recognition based on fitting a 3-D morphable model,IEEE Trans. Pattern Analysis & Machine Intelligence, Vol. 25, No. 9, pp.10631074.
Blanz, V., Grother, P., Phillips, P.J. and Vetter, T. (2005) Face recognition based on frontal viewsgenerated from non-frontal images,IEEE Computer Society Conference on Computer Visionand Pattern Recognition, CVPR 2005, Vol. 2, San Diego, CA, USA, pp.454461.
-
8/2/2019 A Method of Creating 3-D Face Images From 2-D Photos for Face Recognition
16/16
A method of creating 3-D face images from 2-D photos 55
Blanz, V. (2006) Face recognition based on a 3D morphable model, 7th International Conference
on Automatic Face and Gesture Recognition,FGR, 2006, Southampton, UK, pp.617624.Blanz, V., Scherbaum, K. and Seidel, H-P. (2007) Fitting a morphable model to 3D scans
of faces, IEEE 11th International Conference on Computer Vision, ICCV 2007,Rio de Janeiro, Brazil, pp.18.
Etemad, K. and Chellappa, R. (1997) Discriminant analysis for recognition of human faceimages,J. Optical Soc. Am. A, Vol. 14, pp.17241733.
Fu, K.S., Gonzalez, R.C. and Lee, C.S.G. (1987) Robotics, Control, Sensing, Vision, andIntelligence, McGraw-Hill, Inc., pp.2325.
Guan, X. and Zhuang, H. (2007) Application of illumination compensation for face recognition,Proc. Int. Conf. Artificial Intelligence and Pattern Recognition, Orlando, Florida, pp.255261.
Huang, T.S. and Tang, L.A. (1996) 3-D face modeling and its applications, Intl J. PatternRecognition & Artificial Intelligence, Vol. 10, No. 5, pp.491519.
Ishiyama, R. and Sakamoto, S. (2002) Geodesic illumination basis: compensating for illumination
variations in any pose for face recognition, Proceedings 16th International Conference onPattern Recognition, Vol. 4, pp.297301.
Levenberg, K. (1944) A method for the solution of certain problems in least squares, Quart. Appl.Math., Vol. 2, pp.164168.
Liu, C. and Wechsler, H. (2000) Evolutionary pursuit and its application to face recognition,IEEE Trans. Pattern Analysis & Machine Intelligence, Vol. 22, No. 6, pp.570582.
Lowe, D.G. (1991) Fitting parameterized three-dimensional models to images, IEEE Trans.Pattern Analysis & Machine Intelligence, Vol. 13, No. 5, pp.441450.
Marquardt, D. (1963) An algorithm for least squares estimation on nonlinear parameters,SIAM J. Appl. Math., Vol. 11, pp.431441.
Nanavati, S., Thieme, M. and Nanavati, M. (2002)Biometrics, John Wiley & Sons, New York.
Turk, M. and Pentland, A. (1991) Eigenfaces for recognition,J. Cognitive Neuroscience, Vol. 13,No. 1, pp.7186.
SAS Publishing (2007) Principal Component Analysis, http://support.sas.com/publishing/pubcat/chaps/55129.pdf
Swets, D.L. and Weng, J. (1996) Using discriminant eigenfeatures for image retrieval,IEEE Trans. Pattern Analysis & Machine Intelligence, Vol. 18, No. 8, pp.831836.
Ullman, S. and Hutternlocher, D.P. (1990) Recognizing solid objects by alignment with an image,International Journal of Computer Vision, Vol. 5, No. 2, pp.195212.
Wand, M., Fischer, M., Peter, I., Meyer F., Heide, F. and Straer, W. (2001) The randomizedz-buffer algorithm: interactive rendering of highly complex scenes, Proceedings of the 28thAnnual Conference on Computer Graphics and Interactive Techniques, pp.361370.
Watt, A. (2000) 3-D Computer Graphics, 3rd ed., Addison-Wesley, Boston, MA.
Wiskott, L., Fellous, J.M., Kruger, N. and Malsburg, C. (1997) Face recognition by elastic bunchgraph matching, IEEE Trans. Pattern Analysis & Machine Intelligence, Vol. 19, No. 7,pp.775779.
Xiong, H., Swamy, S. and Ahmad, O. (2005) Two-dimensional FLD for face recognition,Pattern Recognition, Vol. 38, pp.11211124.
Yao, P., Evans, G. and Calway, A. (2001) Using affine correspondence to estimate 3-D facialpose,Proceedings. International Conference on Image Processing, Vol. 3, pp.919922.
Zhao, W. and Chellappa, R. (2000) SFS based view synthesis for robust face recognition,Proc. Int'l Conf. Automatic Face and Gesture Recognition, Grenoble, France, pp.285292.