1

Compositional Dictionaries for Domain AdaptiveFace Recognition

Qiang Qiu, and Rama Chellappa, Fellow, IEEE.

Abstract—We present a dictionary learning approach to com-pensate for the transformation of faces due to changes in viewpoint, illumination, resolution, etc. The key idea of our approachis to force domain-invariant sparse coding, i.e., design a consistentsparse representation of the same face in different domains.In this way, classifiers trained on the sparse codes in thesource domain consisting of frontal faces can be applied to thetarget domain (consisting of faces in different poses, illuminationconditions, etc) without much loss in recognition accuracy. Theapproach is to first learn a domain base dictionary, and thendescribe each domain shift (identity, pose, illumination) using asparse representation over the base dictionary. The dictionaryadapted to each domain is expressed as sparse linear combina-tions of the base dictionary. In the context of face recognition,with the proposed compositional dictionary approach, a faceimage can be decomposed into sparse representations for agiven subject, pose and illumination respectively. This approachhas three advantages: first, the extracted sparse representationfor a subject is consistent across domains and enables poseand illumination insensitive face recognition. Second, sparserepresentations for pose and illumination can subsequently beused to estimate the pose and illumination condition of a faceimage. Finally, by composing sparse representations for subjectand the different domains, we can also perform pose alignmentand illumination normalization. Extensive experiments using twopublic face datasets are presented to demonstrate the effectivenessof the proposed approach for face recognition.

Index Terms—Face Recognition, Domain Adaption, SparseRepresentation, Pose Alignment, Illumination Normalization,Multilinear Image Analysis.

I. INTRODUCTION

Many image recognition algorithms often fail while ex-periencing a significant visual domain shift, as they expectthe test data to share the same underlying distribution as thetraining data. A visual domain shift is common and naturalin the context of face recognition. Such domain shift is dueto changes in poses, illumination, resolution, etc.. Domainadaptation [1] is a promising methodology for handling thedomain shift by utilizing knowledge in the source domain forproblems in a different but related target domain. [2] is oneof the earliest works on semi-supervised domain adaptation,where they model data with three underlying distributions:source domain data distribution, target domain data distri-bution and a distribution of data that is common to bothdomains. [3] follows a similar model in handling view pointchanges in the context of activity recognition, where theyassume some activities are observed in both source and targetdomains, while some other activities are only in one of thedomains. Under the above assumption, certain hyperplane-based features trained in the source domain are adapted to the

Q. Qiu is at the Department of Electrical and Computer Engineering, DukeUniversity, Durham, NC 27708 USA. R. Chellappa is with the Center forAutomation Research, UMIACS, University of Maryland, College Park, MD20742 USA (e-mail: qiang.qiu@duke.edu, rama@umiacs.umd.edu)

× × ×

Base dictionary Subject codes Pose codes Illumination codes

=

Same pose Same illumination

Fig. 1: Trilinear sparse decomposition. Given a domain basedictionary, an unknown face image is decomposed into sparserepresentations for each subject, pose and illumination respec-tively. The domain-invariant subject (sparse) codes are used forpose and illumination insensitive face recognition. The poseand illumination codes are also used to estimate the pose andlighting condition of a given face. Composing subject codeswith corresponding domain codes enables pose alignment andillumination normalization. Note that the proposed domain-invariant sparse coding assigns similar subject codes to thefirst and third faces (red and blue), similar pose codes to thefirst and second faces (red and green), and similar illuminationcodes to the second and third faces (green and blue).

target domain for improved classification. Domain adaptationfor object recognition is studied in [4], where the subspacesof the source domain, the target domain and the potentialintermediate domains are modeled as points on the Grassmannmanifold. The shift between domains is learned by exploitingthe geometry of the underlying manifolds. A good survey ondomain adaptation can be found in [4].

Face recognition across domain, e.g., pose and illumination,has proved to be a challenging problem [5], [6], [7]. In[5], the eigen light-field (ELF) algorithm is presented forface recognition across pose and illumination. This algorithmoperates by estimating the eigen light field or the plenopticfunction of the subject’s head using all the pixels of variousimages. In [6], [8], face recognition across pose is performedusing stereo matching distance (SMD). The cost to matcha probe image to a gallery image is used to evaluate thesimilarity of the two images. For near frontal faces, recentface alignment efforts such as [9], [10], [11] have been shownto be effective. For face recognition across severe pose and/orillumination variations, ELF and SMD methods still reportstate-of-the-art results. The proposed compositional dictionarylearning approach shows comparable performance to these twomethods for face recognition across domain shifts due to poseand illumination variations. In addition, our approach can alsobe used to classify the pose and lighting condition of a face,and perform pose alignment and illumination normalization.

2

The approach presented here shares some of the attributesof the Tensorfaces method proposed in [7], [12], [13], but sig-nificantly differs in many aspects. In the Tensorfaces method,face images observed in different domains, i.e., faces imagedin different poses under different illuminations, form a facetensor. Then a multilinear analysis is performed on the facetensor using the N -mode SVD decomposition to obtain acore tensor and multiple mode matrices, each for a differentdomain aspect. The N -mode SVD decomposition is similarto the proposed multilinear sparse decomposition shown inFig. 1, where a given unknown image is decomposed intomultiple sparse representations for the given subject, poseand illumination respectively. However, we show throughexperiments that our method based on sparse decompositionsignificantly outperforms the N -mode SVD decompositionfor face recognition across pose and illumination. Anotheradvantage of the proposed method approach over Tensorfacesis that, the proposed approach provides explicit sparse rep-resentations for each subject and each visual domain, whichcan be used for subject classification and domain estimation.Instead, Tensorfaces performs subject classification throughexhaustive projections and matchings. Another work similarto Tensorfaces is discussed in [14], where a bilinear analysisis presented for face matching across domains. In [14], a 2-mode SVD decomposition is performed.

This paper makes the following main contributions:

• The proposed domain-invariant sparse coding enablesa robust way for multilinear decomposition, and pro-vides explicit sparse representations for out-of-trainingsamples. Note that tensor-based methods obtain repre-sentations for out-of-training samples through exhaustiveprojections and matchings.

• The proposed domain base dictionary learning providesa base dictionary that is independent of subjects and do-mains, and we express the dictionary adapted to a specificdomain as sparse linear combinations of base dictionaryatoms using sparse representation of the domain underconsideration.

• A face image is decomposed into sparse representa-tions for subject, pose and illumination respectively. Thedomain-invariant subject (sparse) codes are used for poseand illumination insensitive face recognition. The poseand illumination codes are also used to estimate the poseand lighting condition of a given face. Composing subjectcodes with corresponding domain codes enables posealignment and illumination normalization.

The remainder of the paper is organized as follows: Sec-tion II discusses some details about sparse decomposition andmultilinear image analysis. In Section III, we formulate thecompositional dictionary learning problem for face recogni-tion. In Section IV, we present the proposed compositionaldictionary learning approach, which consists of algorithms tolearn a domain base dictionary, and perform domain-invariantsparse coding. Experimental evaluations are given in Section Von two public face datasets. Finally, Section VI concludes thepaper.

II. BACKGROUND

A. Sparse Decomposition

Sparse signal representations have recently drawn muchattention in vision, signal and image processing research [15],[16], [17], [18]. This is mainly due to the fact that signals andimages of interest can be sparse in some dictionary. Givenan over-complete dictionary D and a signal y, finding asparse representation of y in D entails solving the followingoptimization problem

x = argminx‖x‖0 subject to y = Dx, (1)

where the `0 sparsity measure ‖x‖0 counts the number ofnonzero elements in the vector x. Problem (1) is NP-hard andcannot be solved in a polynomial time. Hence, approximatesolutions are usually sought [19], [20], [21].

The dictionary D can be either based on a mathematicalmodel of the data or it can be trained directly from the data[22]. It has been observed that learning a dictionary directlyfrom training rather than using a predetermined dictionary(such as wavelet or Gabor) usually leads to better representa-tion and hence can provide improved results in many practicalapplications such as restoration and classification [15], [16].

Various algorithms have been developed for the task oftraining a dictionary from examples. One of the most com-monly used algorithms is the K-SVD algorithm [23]. Let Ybe a set of N input signals in a n-dimensional feature spaceY = [y1...yN ], yi ∈ Rn. In K-SVD, a dictionary with a fixednumber of K items is learned by finding a solution iterativelyto the following problem:

argminD,X‖Y −DX‖2

F

s.t. ∀i, ‖xi‖0 ≤ t (2)

where D = [d1...dK ], di ∈ Rn is the learned dictionary,X = [x1, ...,xN ], xi ∈ RK are the sparse codes of inputsignals Y, and T specifies the sparsity that each signal hasfewer than t items in its decomposition. Each dictionary itemdi is l2-normalized.

B. Multilinear Image Analysis

Linear methods are popular in facial image analysis, suchas principal components analysis (PCA) [24], independentcomponent analysis (ICA) [25], and linear discriminant anal-ysis (LDA) [26]. These conventional linear analysis methodswork best when variations in domains, such as pose andillumination, are not present. When any visual domain isallowed to vary, the linear subspace representation above doesnot capture such variation well.

Under the assumption of Lambertian reflectance, Basri andJacobs [27] showed that images of an object obtained undera wide variety of lighting conditions can be approximatedaccurately with a 9-dimensional linear subspace. [28] utilizesthe fact that 2D harmonic basis images at different posesare related by close-form linear transformations [29], [30],and extends the 9-dimensional illumination linear space withadditional pose information encoded in a linear transformationmatrix. The success of these methods suggests the feasibilityof decomposing a face image into separate representationsfor subject and individual domains, e.g. associated pose andillumination, through multilinear algebra.

3

A multilinear image analysis approach, called Tensorfaces,has been discussed in [7], [12], [13]. Tensor is a multidi-mensional generalization of a matrix. An N -th order tensorD is an N -dimensional matrix comprising N spaces. N -mode SVD, illustrated in Fig. 2, is an extension of SVD thatdecomposes the tensor as the product of N -orthogonal spaces,where Tensor Z , the core tensor, is analogous to the diagonalsingular value matrix in SVD. The mode matrix Un containsthe orthonormal vectors spanning the column space of mode-nflattening of D, i.e., the rearranged tensor elements that forma regular matrix [7].

D

U1

U2

U3

Z

=

Fig. 2: An N -mode SVD (N=3 is illustrated) [7].

Consider the illustrative example presented in [7]. Givenface images of 28 subjects, in 5 poses, 3 illuminations and 3expressions, and each image contains 7943 pixels, we obtain aface tensor D of size 28×5×3×3×7943. Suppose we applya multilinear analysis to the face tensor D using the 5-modedecomposition as (3).D = Z ×Usubject ×Upose ×Uillum ×Uexpre ×Upixels

(3)where the 28 × 5 × 3 × 3 × 7943 core tensor Z governsthe interaction between the factors represented in the 5 modematrices, and each of the mode matrix Un represents subjectsand respective domains. For example, the kth row of the28 × 28 mode matrix Usubject contains the coefficients forsubject k, and the jth row of 5×5 mode matrix Upose containsthe coefficients for pose j.

Tensorfaces perform subject classification through exhaus-tive projections and matchings. In the above examples, fromthe training data, each subject is represented with a 28-sizedvector of coefficients to the 28×5×3×3×7943 basis tensorin (4)

B = Z ×Upose ×Uillum ×Uexpre ×Upixels (4)One can then obtain the basis tensor for a particular pose j,illumination l, and expression e as a 28 × 1 × 1 × 1 × 7943sized subtensor Bj,l,e. The subject coefficients of a givenunknown face image are obtained by exhaustively projectingthis image into a set of candidate basis tensors for every j, l, ecombinations. The resulting vector that yields the smallestdistance to one of the rows in Usubject is adopted as thecoefficients for the subject in the test image. In a similar way,one can obtain the coefficient vectors for pose and illuminationassociated with such a test image.

III. PROBLEM FORMULATION

In this section, we formulate the compositional dictionarylearning (CDL) approach for face recognition. It is noted thatour approach is general and applicable to both image and non-image data. Let Y denote a set of N signals (face images) inan n-dim feature space Y = [y1, ...,yN ], yi ∈ Rn. Giventhat face images are from K different subjects [S1, · · · , SK ],

in J different poses [P1, · · · , PJ ], and under L differentillumination conditions [I1, · · · , IL], Y can be arranged insix different forms as shown in Fig. 4. We assume here thatone image is available for each subject under each pose andillumination, i.e., N = K × J × L.

d1 d2 dK

…

d1

…

d2

dK

vector

transpose

Fig. 3: The vector transpose operator.

A denotes the sparse coefficient matrix of J different poses,A = [a1, ...,aJ ], where aj is the sparse representation forthe pose Pj . Let dim(aj) denote the chosen size of sparsecode vector aj , and dim(aj) ≤ J . B denotes the sparse codematrix of K different subjects, B = [b1, ...,bK ], where bk isthe domain-invariant sparse representation for the subject Sk,and dim(bk) ≤ K. C denotes the sparse coefficient matrixof L different illumination conditions, C = [c1, ..., cL], wherecl is the sparse representation for the illumination conditionIl and dim(cl) ≤ L. The domain base dictionary D containsdim(aj) × dim(bk) × dim(cl) atoms arranging in a similarway as Fig. 4. Each dictionary atom is in the Rn space.

…

…

I1

I2

IL

P1

I1

I2

IL

PJ

…

…

…

…

…

…

…

…

… …

…

…

…

…

… …

…

S1 S2 … SK

…

…

I1

I2

IL

S1

I1

I2

IL

SK

…

…

…

…

…

…

…

…

… …

…

…

…

…

… …

…

P1 P2 … PJ

…

…

S1

S2

SK

P1

S1

S2

SK

PJ

…

…

…

…

…

…

…

…

… …

…

…

…

…

… …

…

I1 I2 … IL

…

…

S1

S2

SK

I1

S1

S2

SK

IL

…

…

…

…

…

…

…

…

… …

…

…

…

…

… …

…

P1 P2 … PJ

…

…

P1

P2

PJ

S1

P1

P2

PJ

SK

…

…

…

…

…

…

…

…

… …

…

…

…

…

… …

…

I1 I2 … IL

…

…

P1

P2

PJ

I1

P1

P2

PJ

IL

…

…

…

…

…

…

…

…

… …

…

…

…

…

… …

…

S1 S2 … SK

Y1 Y2 Y3

Y4 Y5 Y6

Fig. 4: Six forms of arranging face images of K subjects in Jposes under L illumination conditions. Each square denotes aface image in a column vector form.

Any of the six forms in Fig. 4 can be transformed intoanother through a sequence of vector transpose operations.As illustrated in Fig. 3, a vector transpose operation is toconsider (stacked) image vectors in Fig. 4 as values andperform typical matrix transpose operation. For simplicity, wedefine six aggregated vector transpose operations {Ti}6i=1. Forexample, Ti transforms an input matrix, which is in any of thesix forms, into the i-th form defined in Fig. 4 (note that threeout of six operations are actually used).

Let yjlk be a face image of subject Sk in pose Pj under

illumination Il. The dictionary adapted to pose Pj and illumi-nation Il is expressed as

[[DT2aj ]T3cl]

T1 .

yjlk can be sparsely represented using this dictionary as,

yjlk = [[DT2aj ]

T3cl]T1bk,

4

where the subject sparse codes bk are independent of bothPj and Il. In this way, we can represent Fig. 4 in a compactmatrix form as shown in (5).

Y1 = [[DT3C1]T2A1]

T1B1 (5a)

Y2 = [[DT3C2]T1B2]

T2A2 (5b)

Y3 = [[DT1B3]T2A3]

T3C3 (5c)

Y4 = [[DT1B4]T3C4]

T2A4 (5d)

Y5 = [[DT2A5]T1B5]

T3C5 (5e)

Y6 = [[DT2A6]T3C6]

T1B6 (5f)We now provide the details of solutions to the following

two problems• How to learn a base dictionary that is independent of

subject and domains.• Given an input face image and the base dictionary, how

to obtain the sparse representation for the associatedpose and illumination, and the domain-invariant sparserepresentation for the subject.

IV. COMPOSITIONAL DICTIONARY LEARNING

In this section, we first show, given a domain base dictionaryD, sparse coefficient matrices {Ai}6i=1, {Bi}6i=1 and {Ci}6i=1

are equal across different equations in (5). Then, we presentalgorithms to learn a domain base dictionary D, and performdomain-invariant sparse coding.

A. Equivalence of Six FormsTo learn a domain base dictionary D, we first need to

establish the following proposition.

Proposition 1. Given a domain base dictionary D, matrices{Ai}6i=1 in all six equations in (5) are equal, and so arematrices {Bi}6i=1 and {Ci}6i=1.

Proof: First we show matrices Bi in (5a) and (5f) areequal. Y1 and Y6 in Fig. 4 are different only in the roworder. We assume a permutation matrix P16 will permutate therows of Y1 into Y6, i.e., P16Y1 = Y6. Through a dictionarylearning process, e.g., k-SVD [23], we obtain a dictionary D1

and the associated sparse code matrix B1 for Y1. Y1 can bereconstructed as Y1 = D1B1. We change the row order of D1

according to P16 without modifying the actual atom value asD6 = P16D1. We decompose Y6 using D6 as Y6 = D6B6,i.e., P16Y1 = P16D1B6, and we have B1 = B6.

Then we show that matrices Ai, Bi and Ci in (5a) and (5b)are equal. If we stack all the images from the same subjectunder the same pose but different illumination as a single ob-servation, we can consider Y2 = YT

1 . By assuming a bilinearmodel, we can represent Y1 as Y1 = [DcA1]

TB1, and wehave Y2 = YT

1 = [DTc B1]

TA1. As Y2 = [DTc B2]

TA2, Ai

and Bi are equal in (5a) and (5b). As both equations sharea bilinear map DT3Ci, with a common base dictionary D,matrices Ci are also equal in (5a) and (5b).

Finally, we show matrices Ai and Ci in (5a) and (5f) areequal. We have shown in (5a) and (5f) that matrices Bi areequal. [[DT3C1]

T2A1]T1 and [[DT2A6]

T3C6]T1 are different

only in the row order. We can use the bilinear model argument

Input: signals Y, sparsity level Ta, Tb, Tc

Output: domain base dictionary Dbegin

Initialization stage:1. Initialize B by solving (5a) via k-SVDminDb,B‖Y1 −DbB‖2F , s.t. ∀k ‖bk‖o ≤ Tb, where

Db = [[DT3C]T2A]T1

repeat2. apply B to (5a) and solve via k-SVD(B† = BT (BTB)−1)minDa,A

‖(Y1B†)

T2−DaA‖2F , s.t. ∀j ‖aj‖o ≤ Ta,

where Da = [DT3C]T2

3. apply A to (5d) and solve via k-SVDminDc,C‖(Y4A

†)T3 −DcC‖2F , s.t. ∀l ‖cl‖o ≤ Tc,

where Dc = [DT1B]T3

4. apply C to (5e) and solve via k-SVDminDb,B‖(Y5C

†)T1 −DbB‖2F , s.t. ∀k ‖bk‖o ≤ Tb,

where Db = [DT2A]T1

until convergence;5. Design the domain base dictionary:

D← [DT2A]A†;

6. return D;end

Algorithm 1: Domain base dictionary learning.

made above to easily show that matrices Ai and Ci are equalin (5a) and (5f).

Through the transitivity of equivalence, we can further showmatrices Ai in all six equations in (5) are equivalent, and soare matrices Bi and Ci. We drop the subscripts in subsequentdiscussions and denote them as A, B and C.

B. Domain-invariant Sparse CodingAs matrices A, B and C are equal across all six forms in

(5), we propose to learn the base dictionary D using Algorithm1 given below. The domain dictionary learning in Algorithm 1optimizes the following objective function,

minD,A,B,C

‖Y1 − [[DT3C]T2A]T1B‖2F , (6)

s.t. ∀j ‖aj‖o ≤ Ta,∀k ‖bk‖o ≤ Tb,∀l ‖cl‖o ≤ Tc,

where Ta, Tb, and Tc specify the sparsity level, i.e., themaximal number of non-zero values in a sparse vector. Forsimplicity and efficiency, we optimize (6) as a sequence ofdictionary learning subproblems. More specifically, we first letDb = [[DT3C]T2A]T1 , and perform regular sparse dictionarylearning to solve

minDb,B‖Y1 −DbB‖2F , s.t. ∀k ‖bk‖o ≤ Tb.

We then use the obtained B to seek an update to Db tominimize the same error ‖Y1 −DbB‖2F . Taking the deriva-tive with respect to Db, we obtain (Y1 − DbB)BT = 0,leading to the updated Db as Y1B

† = Y1BT (BTB)−1. As

Db = [[DT3C]T2A]T1 , we now use updated Db to obtain Da

and A asminDa,A

‖(Y1B†)

T2 −DaA‖2F , s.t. ∀j ‖aj‖o ≤ Ta,

5

Input: an input face image y, domain base dictionary D,sparsity level Ta, Tb, Tc

Output: sparse representation vector for pose a,illumination c, subject b

beginInitialization stage:1. Initialize domain sparse code vector a and c withrandom values;Sparse coding stage:repeat

2. apply a and c to (5a) and obtain b via OMP,minb‖y − [[D

T3c]T2a]Tb‖22, s.t. ‖b‖o ≤ Tb;

3. apply b and c to (5d) and obtain a via OMP,mina‖y − [[D

T1b]T3c]Ta‖22, s.t. ‖a‖o ≤ Ta;

4. apply a and b to (5e) and obtain c via OMP,minc‖y − [[D

T2a]T1b]T c‖22, s.t. ‖c‖o ≤ Tc;

until convergence;5. return

domain-invariant subject sparse codes: b,pose sparse codes: a,illumination sparse codes: c;

endAlgorithm 2: Domain-invariant sparse coding.

where Da = [DT3C]T2 . Then we fix A to update Da, and

solve for Dc and C, and so on.Algorithm 1 is designed as an iterative method, and each

iteration consists of several typical sparse dictionary learningproblems. Thus, this algorithm is flexible and can rely onany sparse dictionary learning methods. We adopt the highlyefficient dictionary learning method, k-SVD [23]. It is notedthat we can easily omit one domain aspect through dictionary“marginalization”. For example, after learning the base dic-tionary D, we can marginalize over illumination sparse codesmatrix C and adopt [DT3C]T2 as the base dictionary for posedomains only.

With the learned base dictionary D, we can performdomain-invariant sparse coding as shown in Algorithm 2,which minimizes the following objective function for a fixedD,

mina,b,c‖y − [[DT3c]T2a]T1b‖2F , (7)

s.t. ‖a‖o ≤ Ta, ‖b‖o ≤ Tb, ‖c‖o ≤ Tc.

The l0 norm minimization involved here is NP-hard andusually solved using greedy pursuit algorithms, such as basispursuit, orthogonal matching pursuit (OMP) [20], [21], torepresent a signal with the best linear combination of t atomsfrom a dictionary, where t is the sparsity. We adopt OMP inthe paper. OMP is a greedy algorithm that iteratively selectsthe dictionary atom best correlated with the residual; and thenit produces a new approximant by projecting the signal ontoatoms already been selected.

With a base dictionary D learned using Algorithm 1, a faceimage can be decomposed using Algorithm 2 into sparse rep-resentations a for the associated pose and c for illumination,and a domain-invariant sparse representation b for the subject.While minimizing (6) in Algorithm 1, we obtain the learned

domain dictionary D, and model codes A, B, and C. Eachcolumn of A denotes the sparse representation assigned to aparticular pose in the training. When a training pose shownat testing, the decomposed pose code a using Algorithm 2converges to the respective column in A. As shown later,testing poses unseen at training are converged to a sparselinear combination of known poses in a consistent way. Wecan observe similar convergence for both subject codes andillumination codes.

Convergence of Algorithms 1 and 2 can be establishedusing the convergence results of k-SVD discussed in [23].Although both algorithms optimize a single objective function,the convergence depends on the success of greedy pursuitalgorithms involved in each iteration step. We have observedempirical convergence for both Algorithm 1 and 2 in all theexperiments reported below.

During training, the domain dictionary learning consists ofmultiple k-SVD procedures, and the complexity of k-SVD isanalyzed in details in [31]. The complexity of Algorithm 2is more critical, as domain-invariant sparse coding is usuallyperformed at testing. As shown later, it usually takes about10 iterations for Algorithm 2 to converge, and each iterationconsists of three OMP operations. As discussed in [31], [32],different implementations of OMP have different complexities.Considering a signal of dimension m with assumed sparsityt, and a dictionary of N atoms, OMP implemented using theQR Decomposition has the complexity of Nt+mt+ t2.

V. EXPERIMENTAL EVALUATION

f04 f05 f01 f02 f03 f06 f10 f11 f07 f08 f09 f12 f16 f17 f13 f14 f15 f18 f19 f20 f21

(a) Illumination variation

c22 c02

c25

c37 c05

c07

c09

c29 c11 c14 c34

c31

(b) Pose variation

Fig. 5: Pose and illumination variation in the PIE dataset.

This section presents experimental evaluations on two publicface datasets: the CMU PIE dataset [33] and the ExtendedYaleB dataset [34]. The PIE dataset consists of 68 subjectsimaged simultaneously under 13 different poses and 21 light-ing conditions, as shown in Fig. 5. The Extended YaleB datasetcontains 38 subjects with near frontal pose under 64 lightingconditions. 64×48 sized images are used in the domain com-position experiments in Section V-C for clearer visualization.In the remaining experiments, all the face images are resizedto 32 × 24. The proposed Compositional Dictionary learningmethod is refereed to as CDL in subsequent discussions.

Experimental evaluation is summarized as follows: Sec-tion V-A provides the learning configurations for all basedictionaries used. The convergence of domain-invariant sparse

6

coding is illustrated in Section V-B. Section V-C demonstrateshow domain composition is used for pose alignment and illu-mination normalization. Domina-invariant subject representa-tion is adopted for cross-domain recognition in Section V-D.Section V-E shows that the proposed method is more robustfor multilinear decompostion than the tensor-based method.Domain estimation is demonstrated in Section V-F.

A. Learned Domain Base DictionariesIn our experiments, four different domain base dictionaries

D10, D4, D34, and D32 are learned. We explain here theconfigurations for each base dictionary.• D4: This dictionary is learned from the PIE dataset by

using 68 subjects in 4 poses under 21 illumination con-ditions. The four training poses to the dictionary are c02,c07, c09 and c14 poses shown in Fig. 5. The dimensionsof coefficient vectors for subject, pose and illuminationare 68, 4 and 9. The respective coefficient sparsity values,i.e., the maximal number of non-zero coefficients, are 20,4 and 9. Note that there is no defined way to specify thesparsity value, and we manually specify sparsity to makeeach coefficient vector around 1

3 to 12 full.

• D10: This dictionary is learned from the PIE dataset byusing 68 subjects in 10 poses under all 21 illuminationconditions. The three unknown poses to the dictionaryare c27 (frontal), c05 (side) and c22 (profile) poses. Thedimensions of coefficient vectors for subject, pose andillumination are 68, 10 and 9. The respective coefficientsparsity values are 20, 8 and 9.

• D34: This dictionary is learned from the PIE datasetby using the first 34 subjects in 13 poses under 21illumination conditions. The dimensions of coefficientvectors for subject, pose and illumination are 34, 13 and9. The respective coefficient sparsity values are 12, 8 and9.

• D32: This dictionary is learned from the Extended YaleBdataset by using 38 subjects under 32 randomly selectedlighting conditions. The dimensions of coefficient vectorsfor subject and illumination are 38, and 32. The respectivecoefficient sparsity values are 20 and 20.

The choice of the above 4 dictionary configurations isexplained as follows: usually, a common challenging setup forthe PIE dataset is to classify subjects in three poses: frontal(c27), side (c05) and profile (c22). Given 13 poses in PIE,we keep the remaining 10 poses to learn D10; We furtherexperiment with fewer samples, e.g., a subset of the remaining10 poses, to learn D4; Given 68 subjects in PIE, we learn D34

using half of the subjects; Given 64 illumination conditionsin the Extended YaleB data, we learn D32 using half of thelighting conditions.

B. Convergence of Domain-invariant Sparse CodingWe demonstrate here the convergence of the proposed

domain-invariant sparse coding in Algorithm 2 over a basedictionary learned using Algorithm 1. We first learn thedomain base dictionary D10 using Algorithm 1, and alsoobtain the associated domain matrices (learned model codes)A, B and C. The matrix A consists of 10 columns and

each column is a unique sparse representation for one ofthe 10 poses. The matrix B consists of 68 columns, andeach column describes one of the 68 subjects. The matrixC consists of 21 columns, and each column describes oneof the 21 illumination conditions. We observe no significantreconstruction improvements from the learned base dictionaryafter 2 iterations of Algorithm 1.

Given a face image (s43, c29, f05), i.e., subject s43 inpose c29 under illumination f05, Fig. 6a, 6b and 6c showthe decomposed sparse representations for subject s43, posec29 and illumination f05 after 1, 2 and 100 iterations ofAlgorithm 2 respectively. We can notice that the decomposedsparse codes (color red) converge to the learned model codes(color blue) in A, B and C. As shown in Fig. 8a, we observeconvergence after 4 iterations.

[35] proposed a Tensor k-SVD method, which is similarto Tensorfaces but replaces the N -mode SVD with k-SVD toperform multilinear sparse decomposition. Using the Tensor k-SVD method, we are able to learn a Tensor k-SVD dictionaryand the associated domain matrices. As the Tensor k-SVDmethod is designed for data compression, it is not discussedin [35] how to decompose a single image into separate sparsecoefficient vectors over such learned Tensor k-SVD dictionary.We adopt a learned Tensor k-SVD dictionary as the basedictionary for domain-invariant sparse coding using Algo-rithm 2. As shown in Fig.6d, the decomposed sparse codesdo not converge well to the learned model codes. It indicatesthat Algorithm 2 performs an inconsistent decomposition overthe Tensor k-SVD dictionary. Therefore, a base dictionarylearned from Algorithm 1 is required by the proposed domain-invariant sparse coding in Algorithm 2 to enforce a consistentmultilinear sparse decomposition.

We further decompose face images (s43, c27, f13) and(s01, c27, f13) over D10. As shown in Fig. 7, even whenpose c27 is unknown to D10, the decomposed sparse codesfor subjects s43 and s01, and illumination f13 still convergeto the learned models. By comparing the pose codes in Fig. 7aand 7b, we notice that the unknown pose c27 is representedas a sparse linear combination of known poses in a consistentway. Given the non-optimality of the greedy OMP adopted ineach iteration [21], we still observe convergence after about10 iterations for both cases, as shown in Fig.8b and Fig.8c.

C. Domain Composition

Using the proposed trilinear sparse decomposition over abase dictionary as illustrated in Algorithm 2, we extract froma face image the respective sparse representations for subject,pose and illumination. We can then translate a subject to adifferent pose and illumination by composing the correspond-ing subject and domain sparse codes over the base dictionary.As discussed in Sec. II-B, Tensorfaces also enable the de-composition of a face image into separate coefficients for thesubject, pose and illumination through exhaustive projectionsand matchings. We adopt the Tensorfaces method here for afair comparison in our domain composition experiments.

1) Pose Alignment: In Fig. 9a, the base dictionary D34 isused in the CDL experiments. To enable a fair comparison, weadopt the same training data and sparsity values for D34 in

7

0 20 40 60−0.5

0

0.5

1

Subject codes

ModelDecomposed

2 4 6 8 10

0

0.5

1

Pose codes

(Iteration = 1)

ModelDecomposed

2 4 6 8

0

0.5

1

Illumination codes

ModelDecomposed

(a) Sparse codes decomposed over D10 after 1 iteration.

0 20 40 60−0.5

0

0.5

1

Subject codes

ModelDecomposed

2 4 6 8 10

0

0.5

1

Pose codes

(Iteration = 2)

ModelDecomposed

2 4 6 8

0

0.5

1

Illumination codes

ModelDecomposed

(b) Sparse codes decomposed over D10 after 2 iterations.

0 20 40 600

0.2

0.4

0.6

0.8

1

Subject codes

ModelDecomposed

2 4 6 8 100

0.2

0.4

0.6

0.8

1

Pose codes

(Iteration = 100)

ModelDecomposed

2 4 6 8

0

0.5

1

Illumination codes

ModelDecomposed

(c) Sparse codes decomposed over D10 after 100 iterations.

0 20 40 60

0

0.5

1

Subject codes

ModelDecomposed

2 4 6 8 10

0

0.5

1

Pose codes

(Iteration = 100)

ModelDecomposed

2 4 6 8

−0.5

0

0.5

Illumination codes

ModelDecomposed

(d) Sparse codes decomposed over a Tensor k-SVD dictionary [35] after 100 iterations.

Fig. 6: Domain-invariant sparse coding (Algorithm 2) of the face image (s43, c29, f05), i.e., subject s43 in pose c29 andillumination f05, over a domain base dictionary. c29 is a known pose to D10. In (a)-(c), the decomposed sparse codes (colorred) converge to the model codes (color blue) when the base dictionary is learned using Algorithm 1; and such convergenceis not warranted in (d), when the base dictionary is not from Algorithm 1.

the corresponding Tensorfaces experiments. Given faces fromsubject s01 under different poses, where both the subject andposes are present in the training data, we extract the subject(sparse) codes for s01 from each of them. Then we extract thepose codes for c27 (frontal) and the illumination codes for f05from an image of subject s43. It is noted that, for such knownsubject cases, the composition (s01, c27, f05) through bothCDL and Tensorfaces provides good reconstructions to theground truth image. The reconstruction using CDL is clearerthan the one using Tensorfaces.

In Fig. 9b, we first extract the subject codes for s43, whichis an unknown subject to D34. Then we extract the posecodes and the illumination codes from the set of images ofs01 in Fig. 9a. In this unknown subject case, the compositionusing our CDL method provides significantly more accuratereconstruction to the groundtruth images than the Tensorfacesmethod. The central assumption in the literature on sparserepresentation for faces is that the test face image should berepresented in terms of training images of the same subject[36], [37]. As s43 is unknown to D34, therefore, it is expected

that the reconstruction of the subject information is througha linear combination of other known subjects, which is anapproximation but not exact.

In Fig. 9c, the base dictionary D10 is used in the CDLexperiments, and the same training data and sparsity valuesfor D10 are used in the corresponding Tensorfaces experi-ments. We first extract the subject codes for s43. Then weextract the pose codes for pose c22, c05 and c27, whichare unknown poses to the training data. Through domaincomposition, for such unknown pose cases, we obtain moreacceptable reconstruction to the actual images using CDLthan Tensorfaces. This indicates that, using the proposed CDLmethod, an unknown pose can be much better approximatedin terms of a set of observed poses.

2) Illumination Normalization: In Fig. 10a, we use frontalfaces from subject s28, which is known to D34, under differentillumination conditions. For each image, we first isolate thecodes for subject, pose and illumination, and then replace theillumination codes with the one for f11. If f11 is observedin the training data, the illumination codes for f11 can be

8

0 20 40 60

0

0.5

1

Subject codes

ModelDecomposed

2 4 6 8 10

0

0.2

0.4

0.6

0.8

Pose codes

(Iteration = 100)

Decomposed

2 4 6 8

0

0.5

1

Illumination codes

ModelDecomposed

(a) Sparse codes of (s43, c27, f13) decomposed over D10 after 100 iterations.

0 20 40 60

0

0.5

1

Subject codes

ModelDecomposed

2 4 6 8 10

0

0.2

0.4

0.6

0.8

Pose codes

(Iteration = 100)

Decomposed

2 4 6 8

0

0.5

1

Illumination codes

ModelDecomposed

(b) Sparse codes of (s01, c27, f13) decomposed over D10 after 100 iterations.

Fig. 7: Domain-invariant sparse coding (Algorithm 2) of face images (s43, c27, f13) and (s01, c27, f13) over the domain basedictionary D10. c27 is an unknown pose to D10. The decomposed subject and illumination codes (color red) converge to thelearned model codes (color blue). Note that the unknown pose c27 is represented as a sparse linear combination of knownposes in a consistent way.

0 50 1000

0.2

0.4

0.6

Number of Iterations

Dist

ance

to m

odel

cod

es

SubjectPoseIllumination

(a) Sparse decomposition of (s43, c29, f05).

0 50 100

0.1

0.2

0.3

0.4

Number of Iterations

Dist

ance

to m

odel

cod

es

SubjectIllumination

(b) Sparse decomposition of (s43, c27, f13).

0 50 1000

0.2

0.4

0.6

Number of IterationsDi

stan

ce to

mod

el c

odes

SubjectIllumination

(c) Sparse decomposition of (s01, c27, f13).

Fig. 8: Convergence of domain-invariant sparse coding in Algorithm 2.

obtained during training. Otherwise, the illumination codes forf11 can be extracted from a face image of any subjects underf11 illumination. It is shown in Fig. 10a that, for such knownsubject cases, after removing the illumination variation, wecan obtain a reconstructed image close to the ground truthimage using both CDL and Tensorfaces.

Subject s43 in Fig. 10b is unknown to D34. The composedimages from CDL exhibit significantly more accurate subject,pose and illumination reconstruction than Tensorfaces. Asdiscussed before, the reconstruction to the subject here is onlyan approximation but not exact.

D. Pose and Illumination Invariant Face Recognition

1) Classifying PIE 68 Faces using D4 and D10: Fig. 11shows the face recognition performance under combined poseand illumination variation for the CMU PIE dataset. To enablethe comparison with [8], we adopt the same challengingsetup as described in [8]. In this experiment, we classify 68subjects in three poses, frontal (c27), side (c05), and profile(c22), under all 21 lighting conditions. We select one of the3 poses as the gallery pose, and one of the remaining 2poses as the probe pose, for a total of 6 gallery-probe posepairs. For each pose pair, the gallery is under the lightingcondition f11 as specified in [8], and the probe is underthe illumination indicated in the table. Methods compared

here include Tensorface[7], [12], SMD [8], and the proposedmethod CDL. CDL-4 uses the dictionary D4 and CDL-10uses D10. In both CDL-4 and CDL-10 setups, three testingposes c27, c05, and c22 are unknown to the training data. Itis noted that, to the best of our knowledge, SMD reports thebest recognition performance in such experimental setup. Asshown in Fig. 11, among 4 out of 6 Gallery-Probe pose pairs,the proposed CDL-10 is better or comparable to SMD.

The stereo matching distance method performs classificationbased on the stereo matching distance between each pair ofgallery-probe images. The stereo matching method can beseen as an example of a zero-shot method as no training isinvolved. The stereo matching distance becomes more robustwhen the pose variation between such image pair decreases.However, the proposed CDL classifies faces based on subjectcodes extracted from each image alone. The robustness ofthe extracted subject codes only depends on the capability ofthe base dictionary to reconstruct such a face. This explainswhy our CDL method significantly outperforms SMD for morechallenging pose pairs, e.g., Profile-Frontal pair with 62o posevariation; but performs worse than SMD for easier pairs, e.g.,Frontal-Side with 16o pose variation.

It can be observed in Fig. 9c that an unknown pose canbe approximated in terms of a set of observed poses. Byrepresenting three testing poses through four training posesin D4, instead of ten poses in D10, we obtain reasonable

9

(c27, f05) (s01)

+

(s01, c27, f05)

Actual

CDL

T

enso

r

(a) Composition using base dictionary D34. s01 is a known subject to D34. c27 and f05 are extracted from an unknown subject s43.

(s43, f11)

(s43)

+

(c22) (c02) (c37) (c29) (c11) (c14) (f11)

(c22) (c02) (c37) (c29) (c11) (c14)

CD

L T

enso

r

Actual

(b) Composition using base dictionary D34. s43 is an unknown subject to D34.

+

Actual

(s43) (f11)

(c22) (c05) (c27)

(s43, f11) (c22) (c05) (c27)

CD

L

Ten

sor

(c) Composition using base dictionary D10. c22, c05 and c27 are unknown poses to D10.

Fig. 9: Pose alignment through domain composition. In each corresponding Tensorfaces experiment, we adopt the same trainingdata and sparsity values used for the CDL base dictionary for a fair comparison. When a subject or a pose is unknown to thetraining data, the proposed CDL method provides significantly more accurate reconstruction to the ground truth images.

performance degradations but with 60% less training data.Though the Tensorface method shares a similar multilinear

framework to CDL, as seen from Fig. 11, it only handleslimited pose and illumination variations.

2) Classifying Extended YaleB using D32: We adopt asimilar protocol as described in [38]. In the Extended YaleBdataset, each of the 38 subjects is imaged under 64 lightingconditions. We split the dataset into two halves by randomlyselecting 32 lighting conditions as training, and the other halffor testing. Fig. 12 shows the illumination variation in thetesting data. When we learn D32 using Algorithm 1, we alsoobtain one unique domain-invariant sparse representation foreach subject. During testing, we extract the subject codesfrom each testing face image and classify it based on the bestmatch in unique sparse representation of each subject learnedduring training. As shown in Table I, the proposed CDLmethod outperforms other state-of-the-art sparse representationmethods (The results for other compared methods are takenfrom [38]). When the extreme illumination conditions areincluded, we obtain an average recognition rate 98.91%. Byexcluding the extreme illumination condition f35, we obtainan average recognition rate 99.7%.

42

f1 f2 f5 f6 f12 f14 f15 f16 f17 f18 f19 f21 f26 f29 f31 f34

f35 f37 f38 f40 f41 f44' f45 f46 f47 f50 f52 f53 f55 f58 f59 f62

Fig. 12: Illumination variation in the Extended YaleB dataset.

3) Comparisons with More Face Recognition Methods:In this section, we present comparisons with more state-of-the-art face recognition methods to further evaluate theeffectiveness of our approach for face recognition acrossdomains. In [18], a different approach for realizing domain-adaptive face recognition is presented. We adopt the sameexperimental conditions in [18] by learning a domain basedictionary using five training poses c11, c22 c27, c34, andc37. Fig. 13 shows the classification accuracy on 68 subjects5712 face images in four testing poses c02, c05, c14, and c29over 21 different lighting conditions. The proposed method(color red) significantly outperforms state-of-the-art methodsDADL [18] and SRC [36] for face recognition across domains.

We further compare with several techniques designed forillumination robust face representation, including Gradient-

10

Actual

(s28, c27, f11)

(s28, c27)

(f02) (f03) (f04) (f13) (f14)

CD

L

Ten

sor

(a) Composition using base dictionary D34. s28 is a known subject to D34.

Actual

(s43, c27)

(f02) (f03) (f04) (f13) (f14)

(s43, c27, f11)

CD

LT

enso

r

(b) Composition using base dictionary D34. s43 is an unknown subject to D34.

Fig. 10: Illumination normalization through domain composition. In each corresponding Tensorfaces experiment, we adopt thesame training data and sparsity values used for the CDL base dictionary for a fair comparison. When a subject is unknown tothe training data, the proposed CDL method provides significantly more accurate reconstruction to the ground truth images.TABLE I: Face recognition rate (%) on the Extended YaleB face dataset across 32 different lighting conditions. By excludingthe extreme illumination condition f35, we obtain an average recognition rate 99.7%

CDL D-KSVD [39] LC-KSVD [38] K-SVD [23] SRC [36] LLC [40]98.91 94.10 95.00 93.1 97.20 90.7

faces [41], LTV [42], SQI [43], and MSR [44]. Following theexperiments described in [41], we use 68 subjects with 1428frontal (c27) face images, each with 21 different illuminationsfor testing. We use one image per subject as the referenceimages, the other images as the query images. It is noted thatsome of the compared methods here are unsupervised, andthe proposed method requires an additional base dictionarylearning step. We adopt here the domain base dictionary D4

learned from four other training poses. As shown in Table II,the proposed method outperforms compared methods for facerecognition under varying illumination.

As discussed, in Table I, we obtain one unique domain-invariant sparse representation for each subject during thedomain base dictionary learning. During testing, we extractthe subject codes from each testing face image and classifyit based on the best match in unique sparse representation ofeach subject learned from the training data. We now adopt adifferent experimental protocol on the extended YaleB datasetas discussed in [45]. We randomly select 5 images per subjectfrom the training data as the reference and use the remainingimages as the query images. The same base dictionary D32

is adopted. We obtain recognition accuracy 99.80%, which iscomparable to 97.80% reported in [45].

E. Mean Code and Error AnalysisAs discussed in Sec. II-B, the Tensorface method shares a

similar multilinear framework to the proposed CDL method.However, we showed through the above experiments thatthe proposed method based on sparse decomposition signif-icantly outperforms the N -mode SVD decomposition for facerecognition across pose and illumination. In this section, we

analyze in more detail the behaviors of the proposed CDL andTensorfaces, by comparing subject and domain codes extractedfrom a face image using these two methods.

For the experiments in this section, we adopt the basedictionary D10 for CDL, and the same training data andsparsity values of D10 for Tensorfaces to learn the core tensorand the associated mode matrices. The same testing data isused for both methods, i.e., 68 subjects in the PIE dataset under21 illumination conditions in the c27 (frontal), c05 (side) andc22 (profile) poses, which are three unseen poses not presentin the training data.

Fig. 14 and Fig. 15 shows the mean subject codes of subjects1 and s2 over 21 illumination conditions in each of the threetesting poses, and the associated standard errors. In each of thetwo figures, we compare the first row, the subject codes fromCDL, with the second row, the subject codes from Tensorfaces.We can easily notice the following: first, the subject codesextracted using CDL are more sparse; second, CDL subjectcodes are more consistent across pose; third, CDL subjectcodes are more consistent across illumination, which is in-dicated by the smaller standard errors. By comparing Fig. 14with Fig. 15, we also observe that the CDL subject codes aremore discriminative. Table III further shows the square root ofthe pooled variances of subject codes for all 68 subjects over21 illumination conditions in each of the three testing poses.The significantly smaller variance values obtained using CDLindicate the more consistent sparse representation of subjectsdecomposed from face images. Therefore, face recognitionusing CDL subject codes significantly outperforms recognitionusing Tensorfaces subject codes.

11

1 3 5 7 9 11 13 15 17 19 210

0.2

0.4

0.6

0.8

1

Lighting Condition

Reco

gnitio

n Acc

uracy

CDL−10CDL−4TensorfaceSMD

(a) Gallery: profile. Probe: frontal.

1 3 5 7 9 11 13 15 17 19 210

0.2

0.4

0.6

0.8

1

Lighting Condition

Reco

gnitio

n Acc

uracy

CDL−10CDL−4TensorfaceSMD

(b) Gallery: profile. Probe: side.

1 3 5 7 9 11 13 15 17 19 210

0.2

0.4

0.6

0.8

1

Lighting Condition

Reco

gnitio

n Acc

uracy

CDL−10CDL−4TensorfaceSMD

(c) Gallery: frontal. Probe: side.

1 3 5 7 9 11 13 15 17 19 210

0.2

0.4

0.6

0.8

1

Lighting Condition

Reco

gnitio

n Acc

uracy

CDL−10CDL−4TensorfaceSMD

(d) Gallery: frontal. Probe: profile.

1 3 5 7 9 11 13 15 17 19 210

0.2

0.4

0.6

0.8

1

Lighting Condition

Reco

gnitio

n Acc

uracy

CDL−10CDL−4TensorfaceSMD

(e) Gallery: side. Probe: frontal.

1 3 5 7 9 11 13 15 17 19 210

0.2

0.4

0.6

0.8

1

Lighting Condition

Reco

gnitio

n Acc

uracy

CDL−10CDL−4TensorfaceSMD

(f) Gallery: side. Probe: profile.

Fig. 11: Face recognition under combined pose and illumination variations for the CMU PIE dataset. Given three testing poses,Frontal (c27), Side (c05), Profile (c22), we show the percentage of correct recognition for each disjoint pair of Gallery-Probeposes. See Fig. 5 for poses and lighting conditions. Methods compared here include Tensorface [7], [12], SMD [8] and ourcompositional dictionary learning (CDL) method . CDL-4 uses the dictionary D4 and CDL-10 uses D10. To the best of ourknowledge, SMD reports the best recognition performance in such experimental setup. 4 out of 6 Gallery-Probe pose pairs,i.e., (a), (b), (d) and (e), our results are comparable to SMD.

5 10 15 200

0.25

0.5

0.75

1

Lighting Condition

Rec

ogni

tion

Acc

urac

y

CDLDADLEigenfaceSRC

(a) Pose c02

5 10 15 200

0.25

0.5

0.75

1

Lighting Condition

Rec

ogni

tion

Acc

urac

y

CDLDADLEigenfaceSRC

(b) Pose c05

5 10 15 200

0.25

0.5

0.75

1

Lighting Condition

Rec

ogni

tion

Acc

urac

y

CDLDADLEigenfaceSRC

(c) Pose c29

5 10 15 200

0.25

0.5

0.75

1

Lighting ConditionR

ecog

nitio

n A

ccur

acy

CDLDADLEigenfaceSRC

(d) Pose c14

Fig. 13: Face recognition accuracy on the CMU PIE dataset using the experimental protocol in [18]. The domain base dictionaryis learned from five training poses c11, c22 c27, c34, and c37. The classification accuracy is reported on 68 subjects 5712 faceimages in four testing poses c02, c05, c14, and c29 over 21 different lighting conditions. The proposed method is denoted asCDL in color red. The proposed method significantly outperforms state-of-the-art methods DADL [18] and SRC [36] for facerecognition across domains.

TABLE II: Face recognition rate (%) on the PIE face dataset (pose c27) under varying illumination.CDL Gradientfaces [41] LTV [42] SQI [43] MSR [44]99.93 99.83 86.85 77.94 62.07

F. Pose and Illumination Estimation

In Section V-D, we report the results of experiments oversubject codes using base dictionaries D10 and D4. Whilegenerating subject codes, we simultaneously obtain pose codesand illumination codes. Such pose and illumination codes canbe used for pose and illumination estimation. In Fig. 16,we show the pose and illumination estimation performanceon the PIE dataset using the pose and illumination sparsecodes through both CDL and Tensorfaces. The proposedCDL method exhibits significantly better domain estimationaccuracy than the Tensorfaces method. By examining Fig. 16,it can be noticed that the most confusing illumination pairs

in CDL, e.g., (f05, f18), (f10, f19) and (f11, f20) are veryvisually similar based on Fig. 5.

VI. CONCLUSION

We presented an approach to learn domain adaptive dic-tionaries for face recognition across pose and illuminationdomain shifts. With a learned domain base dictionary, anunknown face image is decomposed into subject codes, posecodes and illumination codes. Subject codes are consistentacross domains, and enable pose and illumination insensitiveface recognition. Pose and illumination codes can be usedto estimate the pose and lighting condition of the face. Weplan to evaluate the proposed framework in representing 3D

12

0 10 20 30 40 50 60 70−0.2

0

0.2

0.4

0.6

0.8

1

1.2

Subje

ct Co

de

(a) Frontal pose (c27).

0 10 20 30 40 50 60 70−0.2

0

0.2

0.4

0.6

0.8

1

1.2

Subje

ct Co

de

(b) Side pose (c05).

0 10 20 30 40 50 60 70−0.2

0

0.2

0.4

0.6

0.8

1

1.2

Subje

ct Co

de

(c) Profile pose (c22).

0 5 10 15 20 25−0.6

−0.4

−0.2

0

0.2

0.4

Subje

ct Co

de

(d) Frontal pose (c27).

0 5 10 15 20 25−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

Subje

ct Co

de

(e) Side pose (c05).

0 5 10 15 20 25−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

Subje

ct Co

de

(f) Profile pose (c22).

Fig. 14: Mean subject code of subject s1 over 21 illumination conditions in each of the three testing poses, and standard errorof the mean code. (a),(b),(c) are generated using CDL with the base dictionary D10. (d),(e),(f) are generated using Tensorfaces.

0 10 20 30 40 50 60 70−0.2

0

0.2

0.4

0.6

0.8

1

1.2

Subje

ct Co

de

(a) Frontal pose (c27).

0 10 20 30 40 50 60 70−0.2

0

0.2

0.4

0.6

0.8

1

1.2

Subje

ct Co

de

(b) Side pose (c05).

0 10 20 30 40 50 60 70−0.2

0

0.2

0.4

0.6

0.8

1

1.2

Subje

ct Co

de(c) Profile pose (c22).

0 5 10 15 20 25−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5

Subje

ct Co

de

(d) Frontal pose (c27).

0 5 10 15 20 25−0.5

0

0.5

Subje

ct Co

de

(e) Side pose (c05).

0 5 10 15 20 25−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

Subje

ct Co

de

(f) Profile pose (c22).

Fig. 15: Mean subject code of subject s2 over 21 illumination conditions in each of the three testing poses, and standard errorof the mean code. (a),(b),(c) are generated using CDL with the base dictionary D10. (d),(e),(f) are generated using Tensorfaces.TABLE III: The square root of the pooled variances of subject codes for 68 subjects over 21 illumination conditions in eachof the three testing poses.

Frontal pose (c27) Side pose (c05) Profile pose (c22)CDL 0.0351 0.0590 0.0879Tensorfaces 0.1479 0.1758 0.1814

faces. A face image captured by a RGB-D camera providesprojected 2D images at various poses. Together with synthe-sized light sources, we can construct the proposed domainbase dictionary; and the learned dictionary can then be usedto decompose any given 2D face image for domain-invariantsubject representation. We will also experiment the proposedmethod as a novel way to synthesize more training samplesfrom unseen pose and illumination conditions.

ACKNOWLEDGMENTS

This research was partially supported by a MURI from theOffice of Naval research under the Grant N00014-10-1-0934..

REFERENCES

[1] S. Ben-David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, andJ. Vaughan, “A theory of learning from different domains,” MachineLearning, vol. 79, pp. 151–175, 2010.

[2] H. Daume, III and D. Marcu, “Domain adaptation for statistical classi-fiers,” J. Artif. Int. Res., vol. 26, pp. 101–126, May 2006.

[3] A. Farhadi and M. K. Tabrizi, “Learning to recognize activities from thewrong view point,” in Proc. European Conference on Computer Vision,Marseille, France, Oct. 2008.

[4] R. Gopalan, R. Li, , and R. Chellappa, “Domain adaptation for objectrecognition: An unsupervised approach,” in Proc. Intl. Conf. on Com-puter Vision, Barcelona, Spain, Nov. 2011.

[5] R. Gross, S. Baker, I. Matthews, and T. Kanade, “Face recognition acrosspose and illumination,” in Handbook of Face Recognition. Springer-Verlag, 2004.

13

.59 .21 .09 .02 .00 .00 .00 .00 .02 .00 .00 .00 .00 .00 .00 .00 .03 .02 .00 .00 .00

.20 .50 .10 .02 .00 .01 .00 .00 .08 .00 .00 .00 .00 .00 .00 .01 .04 .01 .00 .01 .00

.10 .13 .51 .03 .02 .00 .00 .00 .04 .00 .00 .00 .00 .00 .00 .01 .12 .01 .01 .00 .00

.01 .02 .06 .56 .05 .01 .00 .01 .05 .00 .00 .00 .00 .00 .00 .00 .07 .10 .02 .00 .00

.00 .00 .01 .06 .54 .05 .01 .00 .05 .03 .00 .00 .00 .00 .00 .00 .01 .15 .06 .00 .00

.00 .01 .00 .01 .05 .79 .03 .00 .03 .01 .00 .00 .00 .00 .00 .00 .00 .00 .01 .00 .00

.00 .00 .01 .00 .00 .03 .77 .06 .00 .04 .03 .00 .00 .00 .00 .00 .00 .01 .01 .01 .00

.00 .00 .00 .00 .00 .00 .04 .76 .00 .00 .03 .05 .05 .00 .00 .00 .00 .00 .00 .01 .02

.02 .08 .05 .03 .05 .07 .00 .00 .64 .00 .00 .00 .00 .00 .00 .00 .01 .02 .00 .00 .00

.00 .00 .00 .01 .02 .03 .02 .00 .01 .56 .04 .00 .00 .00 .00 .00 .02 .01 .22 .03 .00

.00 .00 .00 .00 .00 .00 .02 .03 .00 .04 .60 .03 .01 .01 .00 .00 .00 .00 .03 .19 .01

.00 .01 .00 .00 .00 .00 .00 .05 .00 .00 .01 .63 .10 .05 .01 .00 .00 .00 .00 .00 .11

.00 .00 .00 .00 .00 .00 .00 .04 .00 .00 .00 .09 .62 .05 .00 .01 .00 .00 .00 .04 .13

.00 .00 .00 .00 .00 .00 .00 .00 .00 .00 .00 .04 .04 .64 .11 .01 .01 .00 .00 .00 .11

.00 .01 .01 .00 .00 .00 .00 .00 .01 .00 .00 .01 .01 .09 .73 .08 .00 .00 .00 .00 .04

.00 .01 .00 .01 .00 .00 .00 .00 .00 .00 .00 .00 .01 .03 .16 .75 .00 .00 .00 .00 .02

.03 .03 .13 .08 .02 .01 .00 .00 .02 .01 .00 .00 .00 .01 .00 .00 .59 .04 .00 .00 .00

.01 .02 .00 .12 .14 .00 .01 .00 .03 .01 .00 .00 .00 .00 .00 .00 .02 .58 .04 .00 .00

.00 .00 .00 .00 .06 .00 .00 .00 .00 .19 .03 .00 .00 .00 .00 .00 .00 .02 .65 .02 .00

.00 .00 .00 .00 .00 .00 .01 .01 .00 .02 .17 .00 .06 .01 .00 .00 .00 .00 .03 .64 .03

.00 .00 .00 .00 .00 .00 .00 .01 .00 .00 .01 .08 .11 .12 .03 .02 .00 .00 .00 .01 .57

f1

f2

f3

f4

f5

f6

f7

f8

f9

f10

f11

f12

f13

f14

f15

f16

f17

f18

f19

f20

f21

f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 f14 f15 f16 f17 f18 f19 f20 f21

(a) D10

.72 .14 .06 .02 .00 .01 .00 .00 .01 .00 .00 .00 .00 .00 .00 .00 .02 .01 .00 .00 .00

.11 .57 .12 .02 .00 .03 .00 .00 .04 .01 .00 .00 .00 .00 .00 .00 .05 .01 .00 .00 .00

.02 .12 .51 .03 .01 .01 .00 .00 .05 .02 .00 .00 .01 .00 .00 .00 .16 .01 .00 .00 .00

.00 .00 .04 .45 .02 .02 .00 .00 .13 .01 .01 .00 .00 .00 .00 .00 .14 .13 .01 .01 .00

.00 .00 .01 .04 .52 .11 .02 .00 .01 .02 .00 .00 .00 .00 .00 .00 .01 .22 .01 .00 .00

.00 .01 .00 .02 .13 .59 .06 .01 .06 .02 .01 .00 .00 .00 .00 .00 .00 .05 .01 .00 .00

.00 .00 .00 .00 .02 .03 .75 .07 .01 .07 .02 .01 .00 .00 .00 .00 .00 .00 .01 .00 .00

.00 .00 .00 .00 .00 .00 .08 .72 .00 .01 .04 .07 .02 .00 .00 .00 .00 .00 .00 .02 .01

.01 .03 .04 .15 .01 .10 .01 .02 .46 .01 .00 .00 .00 .00 .01 .00 .08 .04 .00 .00 .00

.00 .00 .00 .01 .02 .01 .06 .00 .01 .48 .03 .00 .00 .00 .00 .00 .00 .01 .33 .02 .00

.00 .00 .00 .00 .00 .00 .01 .05 .00 .04 .53 .03 .04 .00 .00 .00 .00 .01 .05 .21 .01

.00 .00 .00 .00 .00 .00 .00 .04 .00 .00 .01 .72 .11 .02 .02 .00 .00 .00 .01 .01 .03

.00 .00 .00 .00 .00 .00 .00 .02 .00 .00 .02 .14 .50 .04 .01 .00 .00 .00 .01 .05 .18

.00 .00 .00 .00 .00 .00 .00 .00 .00 .00 .00 .02 .05 .59 .10 .06 .00 .00 .00 .00 .15

.01 .00 .01 .00 .00 .00 .00 .00 .00 .00 .00 .00 .02 .09 .69 .11 .00 .00 .00 .00 .04

.01 .00 .01 .00 .00 .00 .00 .00 .00 .00 .00 .00 .00 .04 .08 .81 .00 .00 .00 .00 .02

.01 .01 .17 .12 .00 .00 .00 .00 .11 .00 .00 .00 .00 .00 .00 .00 .49 .02 .02 .01 .00

.01 .00 .00 .08 .25 .07 .01 .00 .03 .01 .01 .00 .01 .00 .00 .00 .01 .44 .03 .00 .00

.00 .00 .00 .00 .01 .01 .03 .00 .00 .28 .05 .01 .00 .00 .00 .00 .01 .01 .54 .02 .00

.00 .00 .00 .00 .00 .01 .00 .03 .00 .01 .21 .04 .08 .01 .00 .00 .00 .00 .02 .54 .01

.00 .00 .00 .00 .00 .00 .00 .00 .00 .00 .01 .02 .15 .13 .03 .03 .00 .00 .00 .03 .56

f1

f2

f3

f4

f5

f6

f7

f8

f9

f10

f11

f12

f13

f14

f15

f16

f17

f18

f19

f20

f21

f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 f14 f15 f16 f17 f18 f19 f20 f21

(b) D4

.45 .20 .09 .06 .01 .01 .00 .00 .03 .01 .00 .00 .01 .00 .00 .02 .08 .01 .00 .00 .00

.17 .30 .15 .04 .02 .01 .00 .00 .05 .00 .00 .02 .00 .00 .03 .02 .12 .02 .00 .00 .00

.08 .12 .25 .11 .02 .02 .00 .00 .08 .01 .00 .01 .01 .02 .00 .00 .20 .02 .01 .00 .01

.02 .06 .07 .17 .10 .05 .00 .00 .15 .02 .00 .01 .00 .00 .00 .00 .15 .10 .04 .03 .00

.00 .02 .03 .07 .15 .13 .04 .04 .06 .07 .01 .00 .02 .00 .00 .00 .05 .17 .06 .02 .00

.01 .02 .02 .05 .11 .26 .10 .03 .11 .05 .02 .02 .00 .00 .00 .00 .04 .06 .03 .01 .01

.00 .00 .00 .01 .03 .09 .33 .07 .03 .12 .07 .05 .03 .01 .00 .00 .00 .03 .06 .02 .01

.00 .00 .00 .01 .01 .02 .03 .34 .01 .05 .10 .13 .08 .03 .01 .01 .00 .00 .01 .07 .04

.03 .04 .08 .09 .06 .11 .04 .02 .25 .03 .01 .01 .01 .01 .00 .00 .10 .05 .02 .00 .01

.01 .00 .01 .01 .03 .04 .10 .08 .02 .20 .04 .03 .02 .03 .01 .01 .02 .03 .21 .04 .01

.00 .00 .00 .00 .02 .02 .05 .09 .00 .04 .21 .08 .10 .06 .01 .01 .00 .01 .04 .16 .08

.00 .01 .00 .00 .00 .01 .03 .13 .00 .01 .05 .30 .13 .08 .04 .00 .01 .00 .02 .07 .08

.00 .00 .01 .00 .02 .00 .01 .05 .01 .02 .08 .13 .26 .08 .02 .01 .00 .00 .02 .07 .16

.00 .00 .01 .00 .00 .00 .01 .00 .01 .00 .04 .09 .10 .29 .10 .05 .02 .00 .00 .05 .18

.00 .03 .02 .00 .00 .00 .00 .02 .00 .00 .02 .03 .04 .08 .47 .17 .00 .01 .04 .01 .05

.03 .04 .00 .00 .00 .00 .00 .01 .00 .00 .00 .01 .01 .06 .17 .58 .01 .00 .01 .01 .03

.04 .06 .18 .13 .04 .03 .00 .00 .09 .01 .00 .02 .01 .01 .00 .00 .28 .07 .01 .01 .01

.01 .02 .04 .11 .15 .08 .03 .01 .05 .05 .02 .02 .01 .01 .00 .00 .09 .19 .06 .02 .00

.00 .00 .01 .04 .05 .04 .06 .03 .01 .21 .07 .02 .03 .01 .04 .02 .01 .04 .24 .03 .01

.01 .00 .00 .01 .00 .01 .02 .10 .00 .05 .21 .07 .06 .05 .02 .02 .00 .01 .03 .25 .06

.00 .00 .00 .00 .00 .00 .01 .03 .01 .00 .07 .12 .19 .14 .03 .02 .01 .01 .01 .02 .29

f1

f2

f3

f4

f5

f6

f7

f8

f9

f10

f11

f12

f13

f14

f15

f16

f17

f18

f19

f20

f21

f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 f14 f15 f16 f17 f18 f19 f20 f21

(c) Tensorfaces

.98 .00 .02

.00 .99 .00

.03 .00 .97

side

profile

front

sideprofile

front

(d) D10

.75 .07 .18

.06 .93 .01

.18 .00 .81

side

profile

front

sideprofile

front

(e) D4

.83 .15 .01

.29 .65 .06

.03 .11 .86

side

profile

front

sideprofile

front

(f) Tensorfaces

Fig. 16: Illumination (a-c) and pose (d-f) estimation on theCMU PIE dataset using base dictionaries D4 and D10. Aver-age accuracy: (a) 0.63, (b) 0.58, (c) 0.28, (d) 0.98, (e) 0.83, (f)0.78. The proposed CDL method exhibits significantly betterdomain estimation accuracy than the Tensorfaces method.

[6] C. Castillo and D. Jacobs, “Wide-baseline stereo for face recognitionwith large pose variation,” in Proc. IEEE Computer Society Conf. onComputer Vision and Patt. Recn., Colorado Springs, CO, June 2011.

[7] M. A. O. Vasilescu and D. Terzopoulos, “Multilinear analysis of imageensembles: Tensorfaces,” in Proc. European Conf. on Computer Vision,Copenhagen, Denmark, 2002.

[8] C. D. Castillo and D. W. Jacobs, “Using stereo matching for 2-d facerecognition across pose,” IEEE Trans. on Patt. Analysis and Mach.Intell., vol. 31, pp. 2298–2304, 2009.

[9] X. Cao, Y. Wei, F. Wen, and J. Sun, “Face alignment by explicit shaperegression,” in IEEE Computer Society Conf. on Computer Vision andPatt. Recn., Providence, Rhode Island, June 2012.

[10] V. Kazemi and J. Sullivan, “One millisecond face alignment with anensemble of regression trees,” in IEEE Computer Society Conf. onComputer Vision and Patt. Recn., Columbus, Ohio, June 2014.

[11] X. Xiong and F. D. la Torre, “Supervised descent method and itsapplication to face alignment,” in IEEE Computer Society Conf. onComputer Vision and Patt. Recn., Portland, Oregon, June 2013.

[12] M. A. O. Vasilescu and D. Terzopoulos, “Multilinear image analysis forfacial recognition,” in Proc. Intl. Conf. on Patt. Recn., Quebec, Canada,Aug, 2002.

[13] S. W. Park and M. Savvides, “Individual kernel tensor-subspaces forrobust face recognition: A computationally efficient tensor frameworkwithout requiring mode factorization,” IEEE Transactions on Systems,Man, and Cybernetics, Part B, vol. 37, no. 5, pp. 1156–1166, 2007.

[14] J. B. Tenenbaum and W. T. Freeman, “Separating style and content withbilinear models,” Neural Computation, vol. 12, pp. 1247–1283, 2000.

[15] R. Rubinstein, A. Bruckstein, and M. Elad, “Dictionaries for sparserepresentation modeling,” Proceedings of the IEEE, vol. 98, no. 6, pp.1045 –1057, June 2010.

[16] J. Wright, Y. Ma, J. Mairal, G. Sapiro, T. Huang, and S. Yan, “Sparserepresentation for computer vision and pattern recognition,” Proceedingsof the IEEE, vol. 98, pp. 1031–1044, June 2010.

[17] Q. Qiu, Z. Jiang, and R. Chellappa, “Sparse dictionary-based repre-sentation and recognition of action attributes,” in Proc. Intl. Conf. onComputer Vision, Barcelona, Spain, Nov. 2011.

[18] Q. Qiu, V. Patel, P. Turaga, and R. Chellappa, “Domain adaptivedictionary learning,” in Proc. European Conference on Computer Vision,Florence, Italy, Oct. 2012.

[19] S. Chen, D. Donoho, and M. Saunders, “Atomic decomposition by basispursuit,” SIAM J. Sci. Comp., vol. 20, pp. 33–61, 1998.

[20] Y. C. Pati, R. Rezaiifar, and P. S. Krishnaprasad, “Orthogonal matchingpursuit: recursive function approximation with applications to waveletdecomposition,” Proc. 27th Asilomar Conference on Signals, Systemsand Computers, Pacific Grove, CA, pp. 40–44, 1993.

[21] J. Tropp, “Greed is good: algorithmic results for sparse approximation,”IEEE Trans. on Information Theory, vol. 50, pp. 2231 – 2242, 2004.

[22] B. A. Olshausen and D. J. Field, “Emergence of simple-cell receptivefield properties by learning a sparse code for natural images,” Nature,vol. 381, no. 6583, pp. 607–609, 1996.

[23] M. Aharon, M. Elad, and A. Bruckstein, “k-SVD: An algorithm fordesigning overcomplete dictionaries for sparse representation,” IEEETrans. on Signal Processing, vol. 54, pp. 4311 –4322, Nov. 2006.

[24] M. Turk and A. Pentland, “Face recognition using eigenfaces,” in Proc.IEEE Computer Society Conf. on Computer Vision and Patt. Recn.,Maui, Hawaii, June 1991.

[25] R. Chellappa, C. Wilson, and S. Sirohey, “Human and machine recogni-tion of faces: A survey,” Proceedings of the IEEE, vol. 83, pp. 705–740,May 1995.

[26] P. Belhumeur, J. Hespanha, and D. Kriegman, “Eigenfaces vs. fisher-faces: Recognition using class specific linear projection,” IEEE Trans.on Patt. anal. and Mach. Intell., vol. 19, pp. 711–720, July 1997.

[27] R. Basri and D. W. Jacobs, “Lambertian reflectance and linear sub-spaces,” IEEE Trans. on Patt. anal. and Mach. Intell., vol. 25, pp. 218–233, February 2003.

[28] Z. Yue, W. Zhao, and R. Chellappa, “Pose-encoded spherical harmonicsfor face recognition and synthesis using a single image,” EURASIPJournal on Advances in Signal Processing, vol. 2008, no. 65, January2008.

[29] R. Ramamoorthi and P. Hanrahan, “A signal-processing framework forreflection,” ACM Transactions on Graphics, vol. 23, pp. 1004–1042, Oct2004.

[30] Y. Tanabe, T. Inui, and Y. Onodera, Group Theory and Its Applicationsin Physics. Springer, Berlin, Germany, 1990.

[31] R. Rubinstein, M. Zibulevsky, and M. Elad, “Efficient implementationof the k-svd algorithm using batch orthogonal matching pursuit,” 2008,Technical Report - CS Technion.

[32] B. Sturm and M. Christensen, “Comparison of orthogonal matchingpursuit implementations,” in 2012 Proceedings of the 20th EuropeanSignal Processing Conference, Aug. 2012.

[33] T. Sim, S. Baker, and M. Bsat, “The CMU pose, illumination, andexpression (PIE) database,” IEEE Trans. on Patt. Anal. and Mach. Intell.,vol. 25, pp. 1615 –1618, Dec. 2003.

[34] A. S. Georghiades, P. N. Belhumeur, and D. J. Kriegman, “From fewto many: Illumination cone models for face recognition under variablelighting and pose,” IEEE Trans. on Patt. Anal. and Mach. Intell., vol. 23,pp. 643–660, June 2001.

[35] R. Ruiters and R. Klein, “Btf compression via sparse tensor decom-position,” Computer Graphics Forum, vol. 28, no. 4, pp. 1181–1188,2009.

[36] J. Wright, A. Yang, A. Ganesh, S. Sastry, and Y. Ma, “Robust facerecognition via sparse representation,” IEEE Trans. on Patt. Anal. andMach. Intell., vol. 31, pp. 210–227, Feb. 2009.

[37] H. Zhang, J. Yang, Y. Zhang, and T. Huang, “Close the loop: Joint blindimage restoration and recognition with sparse representation prior,” inProc. Intl. Conf. on Computer Vision, Barcelona, Spain, Nov. 2011.

[38] Z. Jiang, Z. Lin, and L. S. Davis, “Learning a discriminative dictionaryfor sparse coding via label consistent k-svd,” in Proc. IEEE ComputerSociety Cnf. on Computer Vision and Patt. Recn., Colorado springs, CO,June 2011.

[39] Q. Zhang and B. Li, “Discriminative k-svd for dictionary learning inface recognition,” in Proc. IEEE Computer Society Conf. on ComputerVision and Patt. Recn., San Francisco. CA, June 2010.

[40] J. Wang, J. Yang, K. Yu, F. Lv, T. Huang, and Y. Gong, “Locality-constrained linear coding for image classification,” in Proc. IEEE Com-puter Society Conf. on Computer Vision and Patt. Recn., San Francisco,CA, June 2010, pp. 3360–3367.

[41] T. Zhang, Y. Y. Tang, B. Fang, Z. Shang, and X. Liu, “Face recognitionunder varying illumination using gradientfaces,” IEEE Trans. on ImageProcessing, vol. 18, no. 11, pp. 2599–2606, 2009.

[42] T. Chen, W. Yin, X. Zhou, D. Comaniciu, and T. S. Huang, “Totalvariation models for variable lighting face recognition,” IEEE Trans. onPatt. anal. and Mach. Intell., vol. 28, no. 9, pp. 1519 – 1524, 2006.

[43] H. Wang, S. Z. Li, and Y. Wang, “Generalized quotient image,” in Proc.IEEE Computer Society Conf. on Computer Vision and Patt. Recn.,Washington, DC, June 2004.

[44] D. J. Jobson, Z. Rahman, and G. A. Woodell, “A multi-scale retinex forbridging the gap between color images and the human observation ofscenes,” IEEE Trans. on Image Processing, vol. 6, no. 7, pp. 965–976,1997.

[45] G. Tzimiropoulos, S. Zafeiriou, and M. Pantic, “Subspace learning fromimage gradient orientations,” IEEE Trans. on Patt. anal. and Mach.Intell., vol. 34, no. 12, pp. 2454–2466, 2012.

14

Qiang Qiu received his Bachelor’s degree with firstclass honors in Computer Science in 2001, and hisMaster’s degree in Computer Science in 2002, fromNational University of Singapore. He received hisPh.D. degree in Computer Science in 2012 fromUniversity of Maryland, College Park. During 2002-2007, he was a Senior Research Engineer at Institutefor Infocomm Research, Singapore. He is currentlya Postdoctoral Associate at the Department of Elec-trical and Computer Engineering, Duke University.His research interests include computer vision and

machine learning, specifically on face recognition, human activity recognition,image classification, and sparse representation.

Rama Chellappa received the B.E. (Hons.) degreein Electronics and Communication Engineering fromthe University of Madras, India and the M.E. (withDistinction) degree from the Indian Institute of Sci-ence, Bangalore, India. He received the M.S.E.E. andPh.D. Degrees in Electrical Engineering from PurdueUniversity, West Lafayette, IN. During 1981-1991,he was a faculty member in the department of EE-Systems at University of Southern California (USC).Since 1991, he has been a Professor of Electricaland Computer Engineering (ECE) and an affiliate

Professor of Computer Science at University of Maryland (UMD), CollegePark. He is also affiliated with the Center for Automation Research andthe Institute for Advanced Computer Studies (Permanent Member) and isserving as the Chair of the ECE department. In 2005, he was named aMinta Martin Professor of Engineering. His current research interests spanmany areas in image processing, computer vision and pattern recognition.Prof. Chellappa has received several awards including an NSF PresidentialYoung Investigator Award, four IBM Faculty Development Awards, two paperawards and the K.S. Fu Prize from the International Association of PatternRecognition (IAPR). He is a recipient of the Society, Technical Achievementand Meritorious Service Awards from the IEEE Signal Processing Society.He also received the Technical Achievement and Meritorious Service Awardsfrom the IEEE Computer Society. He is a recipient of Excellence in teachingaward from the School of Engineering at USC. At UMD, he receivedcollege and university level recognitions for research, teaching, innovationand mentoring undergraduate students. In 2010, he was recognized as anOutstanding ECE by Purdue University. Prof. Chellappa served as the Editor-in-Chief of IEEE Transactions on Pattern Analysis and Machine Intelligenceand as the General and Technical Program Chair/Co-Chair for several IEEEinternational and national conferences and workshops. He is a Golden CoreMember of the IEEE Computer Society, served as a Distinguished Lecturer ofthe IEEE Signal Processing Society and as the President of IEEE BiometricsCouncil. He is a Fellow of IEEE, IAPR, OSA, AAAS, ACM and AAAI andholds four patents.