statistical properties of color-signal spaces

1OshsRatsta

tntvlmwwlwtgcanvostTpsegmt

820 J. Opt. Soc. Am. A/Vol. 22, No. 5 /May 2005 R. Lenz and T. H. Bui

Statistical properties of color-signal spaces

Reiner Lenz and Thanh Hai Bui

Department of Science and Technology, Linköping University Bredgatan, SE-60174 Norrköping, Sweden

Received October 5, 2004; accepted November 10, 2004

In applications of principal component analysis (PCA) it has often been observed that the eigenvector with thelargest eigenvalue has only nonnegative entries when the vectors of the underlying stochastic process haveonly nonnegative values. This has been used to show that the coordinate vectors in PCA are all located in acone. We prove that the nonnegativity of the first eigenvector follows from the Perron–Frobenius (and Krein–Rutman theory). Experiments show also that for stochastic processes with nonnegative signals the mean vec-tor is often very similar to the first eigenvector. This is not true in general, but we first give a heuristical ex-planation why we can expect such a similarity. We then derive a connection between the dominance of the firsteigenvalue and the similarity between the mean and the first eigenvector and show how to check the relativesize of the first eigenvalue without actually computing it. In the last part of the paper we discuss the implica-tion of theoretical results for multispectral color processing. © 2005 Optical Society of America

OCIS codes: 330.1690, 000.3860.

lmr

teppttaftahpfitvecvgawoe

ASDni(scfiw

. INTRODUCTIONne attempt to understand the properties of biological

ystems (and to build technological systems) relies on theypothesis that successful systems are adapted to thetructure of the space of signals they are analyzing (seeefs. 1–4 for a few examples and Ref. 5 for a collection ofrticles on natural stimulus statistics.). If this is true,hen it is of interest to investigate properties of signalpaces that are often analyzed by natural or artificial sys-ems. In this paper we investigate spaces of color spectras a typical and important example.Our starting point is the observation that in many in-

eresting cases the signals of interest can assume onlyonnegative values. A typical example is illuminant spec-ra. Here the signals s are functions of the wavelengthariable l and ssld is the number of photons of wave-ength l. By definition ssldù0, ∀l. Another example from

ultichannel color processing are reflectance spectra rhere rsld describes the probability that a photon ofavelength l will be reflected from a point. When an il-

umination with spectrum s interacts with an object pointith reflection spectrum r, then (in the simplest model)

he spectrum that is reflected from that scene point isiven by c=sr, and c is often called a color signal. Also theolor signal is by definition a nonnegative function. Thesere typical examples of the general case in which the sig-al describes counts, probabilities, or other positive-alued quantities. In the following we use the terminol-gy of multichannel color processing and use the termpectrum as a synonym for nonnegative signals. However,he results are valid for nonnegative signals in general.he main result that we will derive in this paper is theroof that all components of the first eigenfunction of atochastic process of spectra have the same sign. The firstigenfunction can therefore be chosen to have only nonne-ative entries. This phenomenon has been observed inany empirical investigations of databases of spectra, but

o our knowledge it has never been pointed out that it fol-

1084-7529/05/050820-8/$15.00 © 2

ows from the Perron–Frobenius theory of nonnegativeatrices and its generalization, the Krein–Rutman theo-

em.In Section 2 we will investigate a few relations among

he mean, the eigenfunctions of the correlation, and theigenfunctions of the covariance operators of stochasticrocesses of nonnegative signals. We will show that forrocesses with color signals the mean, the first eigenvec-or of the correlation matrix, and the first eigenvector ofhe covariance matrix are very similar. However, we willlso give a counterexample in which these vectors are dif-erent, showing that the similarity between the mean andhe first eigenvector is not necessary but that it is a char-cteristic property of all databases of color signals that weave investigated so far. In an attempt to understand thisroperty, we link the similarity to the dominance of therst eigenvalue, an effect that has also been observed of-en in experimental studies (we found that the first eigen-alue could account for more than 90% of the sum of alligenvalues). From this connection we derive a test thatan be used to estimate the dominance of the first eigen-alue without actually computing the eigenvectors or ei-envalues. We will illustrate the discussed properties fornumber of relevant databases of color signals. Finally,e will demonstate some important consequences for self-

rganizing systems such as the low-level filters used inarly vision processing.

. Principal Component Analysis and Nonnegativeignalsescribing the signals of a stochastic process by a fixedumber of coefficients such that the mean squared error

s minimized is known as principal component analysisPCA). These coefficients are obtained by expanding theignals in the basis spanned by the eigenfunctions of theorrelation operator. In the following we show that therst eigenvector (eigenfunction) of a stochastic processith nonnegative signals is also a nonnegative vector

005 Optical Society of America

(dqmcvmtwpttswtt

BFtFft

g

t

tnfithmtlcco

WCca=

n(

t

c

a

rAtt

mre

dvtf

adstt

w(stnnthtct

cot

Sae

CWcpwbo

Psh2

R. Lenz and T. H. Bui Vol. 22, No. 5 /May 2005/J. Opt. Soc. Am. A 821

function). In Subsection 1.B discuss the finite-imensional case and show that this is a simple conse-uence of the Perron–Frobenius theory of nonnegativeatrices. For practical applications this result is suffi-

ient since the signals used there are always described byectors. However, such vectors are always the result ofeasurements, and therefore they depend on the proper-

ies of the measurement device used to produce them. Ife want to study the properties of the color signals inde-endent of the measurement device used to observe them,hen the simple vector space approach is not sufficient. Inhis case a Hilbert-space framework (such as the one de-cribed in Ref. 6) is more appropriate. In Subsection 1.Ce will therefore prove the same fact for processes where

he signals are functions. This proof is based on a form ofhe Krein–Rutman theorem.

. Finite-Dimensional Signalsor finite-dimensional signal vectors the nonnegativity ofhe first eigenvector follows easily from the Perron–robenius theory of nonnegative matrices. We will there-

ore give a brief overview and refer the interested readero Chap. 13 of Ref. 7 for more details.

Definition 1.1. A matrix C is nonnegative if all elements are nonne-

ative.2. A matrix C is positive if all elements are positive.3. A matrix C is reducible if there is a permutation ma-

rix P such that

P−1AP = P8AP = F C1 0

C21 C2G . s1d

A matrix that is not reducible is called irreducible.

Note that all positive matrices are irreducible and thathere is a difference between nonnegative (positive) andonnegative (positive) definite matrices. The first are de-ned by properties of the elements of the matrix, whereashe latter are defined via bilinear products. Note also thatere we require the transformation matrix P to be a per-utation matrix, and for a permutation matrix P the

ransposed P8 of the matrix is its inverse P−1. In the fol-owing we consider only correlation or covariance matri-es. These matrices are symmetrical, and in this specialase we see that a symmetrical matrix is reducible if andnly if it is block-diagonal, i.e., of the form

P−1AP = P8AP = FC1 0

0 C2G . s2d

e write block-diagonal matrices with M blocks1,… ,CM as diag fC1 ,… ,CMg. Iterating this procedure we

an show that for each symmetrical matrix A we can findpermutation matrix such that P−1AP=P8AP

diagfC1 ,… ,CMg with irreducible matrics C1 ,… ,CM.The main result of the Perron–Frobenius theory of non-

egative matrices is the following theorem of Frobeniussee Ref. 7, p. 398):

Theorem 1 (Frobenius). A nonnegative, irreducible ma-rix C has the following properties:

1. There is a simple, real characteristic root r.0 of theharacteristic equation.

2. The absolute value of all the other roots of the char-cteristic equation is less or equal to r.3. The eigenvector belonging to the maximal eigenvalue

has positive coordinates.special case is the theorem of Perron about the exis-

ence of a positive maximal eigenvector of a positive ma-rix.

Theorem 2 (Perron). A positive matrix has a real, maxi-al, positive eigenvalue r. This eigenvalue is a simple

oot of the characteristic equation, and the correspondingigenvector has only positive elements.

For a nonnegative reducible symmetrical matrix of typeiagfC1 ,… ,CMg it follows that there are M positive eigen-alues with nonnegative eigenvectors. The structure ofhese eigenvectors (i.e., the sections with positive entries)ollows the structure of the block-diagonals.

For a stochastic process assuming values ssvd[Rn inn n-dimensional vector space the correlation matrix C isefined as C=Esss8d, where Es.d denotes the mean with re-pect to the stochastic variable v. For future use we recallhat the covariance matrix S is the correlation matrix ofhe centered stochastic variable:

S = E„ss − mdss − md8… = Esss8d − mm8 = C − mm8, s3d

here m is the mean vector. For nonnegative vectors swith elements siù0,1ø iøn) the correlation matrix isymmetric and nonnegative. The correlation matrix isherefore (equivalent to) a block-diagonal matrix withonnegative irreducible square matrices along the diago-al. A stochastic process with nonnegative signals is thushe sum of uncorrelated processes where each subprocessas an irreducible correlation matrix. It is thus sufficiento consider only nonnegative processes with irreducibleorrelation matrices. The main result is the followingheorem:

Theorem 3. The correlation matrix of a stochastic pro-ess with finite-dimensional positive signals has exactlyne eigenvector with only positive entries. This eigenvec-or belongs to the largest eigenvalue.

ince correlation matrices computed from color spectrare usually irreducible, we find that in this case the firstigenvector has only positive entries.

. Function Spacese now derive the corresponding result for stochastic pro-

esses with values in Hilbert spaces. Apart from the inde-endence of the measurement device mentioned earlier,e will also see that the Krein–Rutman theory provides aetter insight into the importance of the conical geometryf these signal spaces.

The Krein–Rutman theorem is a generalization of theerron–Frobenius theory to general Banach (and Hilbert)paces. There are many variations of the basic result, andere we mainly follow the description given in Ref. 8 (p.129); but see also Refs. 9 and 10. We first introduce some

dotùnpTtc

dpdc

TswvtmLtt

toap

iTsoffcfpnl

sr=rtt

Ir

teuopg

ptlf

2OTchslwfseocfi

sttfwtxmbatgvsuuvosstdweltAtealabsirt

t


efinitions: We say a vector space is ordered if there is anrder relation ù that satisfies the following two condi-ions: (a) if fùg, then f+hùg+h, and (b) if fùg, then gf

gg for all elements f, g, h in the vector space and allonnegative numbers g. The subset K= hfù0j is called theositive cone. A linear mapping T is called positive ifsfdù0 for all fù0. In the following we will also use posi-

ive definite operators, and we recall that an operator T isalled positive definite if kTf , fl.0 for all f.

For positive operators we have the following theorem:

Theorem 4 (Krein–Rutman). Assume that X is an or-ered real Banach space such that spsKd=X. If T is a com-act, linear, positive operator with positive spectral ra-ius rsTd, then rsTd is an eigenvalue and has aorresponding positive eigenvector.

he condition spsKd=X means that the closure of thepace of linear combinations of elements in K spans thehole space. The interested reader can find a brief over-iew of the definitions of a Banach space, compact opera-ors, and spectral radius in Appendix A, and more infor-ation can be found in any book on functional analysis.11

ater we will see that in our applications all the condi-ions are fulfilled and we always have the stronger condi-ion that X= hf−g , f ,g[Kj.

We now apply the Krein–Rutman theorem to signalshat are elements of a Hilbert space. We start with a shortverview of the theory of PCA in the Hilbert space contextnd introduce some notation used in the rest of theaper.6,12

We consider the wavelength interval I= slmin,lmaxd. Its not required that I be an interval; it could be some set.he space „L2sId , k,l… is the usual Hilbert space with itscalar product kf ,gl and norm ifi2= kf , fl. We introduce therder relation ù pointwise: fùg if and only if fsldùgsldor all l[I. The cone K is the subset of all nonnegativeunctions: K= hf : fù0j. A spectrum is an element in theone and thus has nonnegative values everywhere. Everyunction can be decomposed into a positive and a negativeart, and therefore L2=K−K. With these notations we canow summarize the Hilbert-space version of PCA as fol-

ows.Consider spectra as results of a stochastic process:

vsld=ssl ,vd, where v is the stotchastic variable. The cor-elation function C is the function Csl1 ,l2dE„svsl1dsvsl2d…, where Es.d denotes the expectation withespect to the stochastic variable v. This function defineshe correlation operator AC that maps functions f to func-ions fC via

ACfsl2d = fCsl2d = kCsl1,l2d,fsl1dl. s4d

n this scalar product C is a function of l1 (and l2 a pa-ameter).

For processes of positive signals the correlation func-ion and the correlation operator have the following prop-rties: The correlation function Csl1 ,l2d has positive val-es and is symmetrical „Csl1 ,l2d=Csl2 ,l1d…. Theperator AC is compact, self-adjoint, positive definite, andositive. Since AC is self-adjoint, it follows that all its ei-envalues are real, and they are all positive since it is also

ositive definite. Finally, we apply the Krein–Rutmanheorem and find that the eigenfunction belonging to theargest eigenvalue is an element of the cone K and there-ore positive.

. STATISTICAL PROPERTIES OF SPACESF COLOR SIGNALS

he fact that the first eigenfunction of a stochastic pro-ess of spectra assumes in most cases only positive valuesas profound consequences for the shape of spaces ofpectra. From the positivity of the first eigenvector fol-ows that the signal space has a conical structure. Thisas investigated in Refs. 13 and 14 (see also Refs. 15–17

or other investigations of conical structures of colorpaces). In this paper we will not investigate these prop-rties (although the Krein–Rutman theory provides an-ther strong link to conical structures) but instead willoncentrate on the relation between the mean and therst eigenvector.The basic observation is the following: Consider the

ubspace L02 of L2 spanned by the first eigenfunction b0. In

he finite-dimensional case, assume also that the correla-ion matrix is irreducible and the first eigenvector there-ore positive everywhere. In the Hilbert-space frameworke assume positivity for simplicity. This defines an or-

hogonal decomposition L2=L02

% L'2 ; i.e., every element

[L2 has a decomposition x= kx ,b0lb0+ kx ,b'lb'. All ele-ents in L'

2 and especially b' are orthogonal to b0. Since0 has positive values everywhere, it follows that b' mustssume both positive and negative values. Now considerhe mean m over all spectra. Since all spectra are nonne-ative valued, it follows that m also has only nonnegativealues. In the decomposition m= km ,b0lb0+ km ,b'lb' weee that km ,b0l.0, and since positive and negative val-es compensate each other, we can also expect that

km ,b'lu! km ,b0l. The (normed) mean and the first eigen-ector should therefore be very similar. Empirically it hasften been observed that the mean of collections of colorpectra and the corresponding first eigenfunction are veryimilar, and we see that the theoretical explanation ofhis fact lies in the Krein–Rutman theory. This heuristicalerivation, however, is not a proof, and counterexampleshere the mean and the first eigenvector are very differ-

nt can be easily constructed. A very simple example il-ustrating this is the following: Consider a “blue” spec-rum defined as s1 ,0d and the “red” spectrum as s0 ,1d.ssume that blue has a probability p. Then the mean vec-

or is sp ,1−pd and the correlation matrix is f p0

01−p

g. Theigenvectors are the unit vectors and obviously in generalre very different from the mean. The fact that the corre-ation matrix is reducible can be easily avoided by adding

small component to the zero channel of the red or thelue spectrum and does not influence our argument. Theimilarity between the mean and the first eigenvector ismportant, however, and is almost always observed foreal color spaces. We will therefore investigate this rela-ion further in our experiments.

In the simple example with the two-dimensional spec-ra, we see that for high or low values of p the eigenvec-

tsc

ddnbanm

w

St

F=lase

stavm

wtadio

a

Iao

iT=

avbwwFgtqo

mttgcmvvf<

ab

Tampdsttfttm


ors and the mean vector become similar. We will nowhow that this is a general property of nonnegative sto-hastic processes.

In the following we will consider only the finite-imensional case, and since all the arguments will be in-ependent of the basis used, we will consider only diago-al correlation matrices. We denote the eigenvector basisy b0 ,b1 ,… ,bk with corresponding (sorted) eigenvalues0ùa1¯ùak. We write a color signal s as linear combi-ation of the eigenvectors and get for the signal and theean

s = ok=0

K

skbk, m = Essd = ok=0

K

mkbk, s5d

ith mk=Esskd.For the correlation matrix C we get

C = ESok=0

K

skbkol=0

K

slbl8D = ok=0

K

ol=0

K

Essksldbkbl8. s6d

ince the bk are eigenvectors of C, we have Cbk=akbk andherefore

Essksld = akdkl. s7d

rom this we see that if ak=Essk2d is small, then mk

Esskd is also small. If we now show that a0 is mucharger than a1, then we know that m1=Ess1d is very smallnd from a0ùa1ù¯ùaK follows that all other mk will bemall. In that case the scaled mean is similar to the firstigenvector.

From the previous argument it follows that if we canhow that the first eigenvalues dominates the others,hen the mean is similar to the first eigenvector (but it islso easy to show in processes with more balanced eigen-alue distributions, there can be similarity between theean and the first eigenvector).We will now derive a method that allows us to check

hether the first eigenvalue is indeed dominating. Forhis we introduce the scaled correlation matrix (we can doll the following computations in the coordinate systemefined by the eigenvectors of the correlation matrix, andt is therefore sufficient to consider diagonal matricesnly):

N =1

TracesCdC = 3

x0 0 . 0

0 x1 . 0

0 . . 0

0 0 . xK

4 s8d

nd its square,

N2 = 3x0

2 0 . 0

0 x12 . 0

0 . . 0

0 0 . xK24 . s9d

We introduce the variables g, S, and get

S = ok=2

K

xk, s10d

1 = ok=0

K

xk = x0 + x1 + S, s11d

g = TracesN2d = ok=0

K

xk2 = s1 − x1 − Sd2 + x1

2 + ok=2

K

xk2. s12d

n the following we assume that the value of g is givennd that we want to investigate the properties of the setf all solutions parameterized by sx2 ,… ,xKd.

The first property that we want to show is that increas-ng the value of xm ,m.1 leads to a decreasing value of x1.he maximum value of x1 is therefore obtained when xm0. Since we consider only solutions, we find that

0 =]g

]xm= − 2s1 − x1 − Sd + 2xm, s13d

nd therefore x1=1−S−xm=1−ok=2K xk−xm. Increasing the

alue of the variable xm and keeping all others fixed cane achieved only by decreasing the value of x1. From thise conclude that the maximum value of x1 is obtainedhen all the other variables are zero: x2=x3=¯=xK=0.rom the trace condition we find that for a given value ofthe maximum value of the largest eigenvalue is a solu-

ion of g=x02+ s1−x0d2=1−2x0+2x0

2 (the other root of theuadratic equation is excluded since the eigenvalues arerdered and therefore x0.1−x0):

x0 =1 + Î2g − 1

2. s14d

Next we compare the first eigenvector of the correlationatrix, the first eigenvector of the covariance matrix, and

he mean (see Refs. 18–20 for related comments). Sincehe mean is removed from the stochastic variable, it is ineneral unlikely that the covariance matrix [see Eq. (3)]ontains information about the mean. In all of our experi-ents with color signals we see, however, that the eigen-

ectors of the correlation and the covariance matrix areery similar. One explanation of this observation is theollowing: In the case where the approximation m

km ,b0lb0 holds, we get

S = C − mm8 < C − km,b0l2b0b08 , s15d

nd since b0 is the first eigenvector of C with eigenvalue0, we get

Sb0 = Cb0 − mm8b0 < b0b0 − km,b0l2b0 = sb0 − km,b0l2db0.

s16d

he first eigenvector of the correlation matrix is thereforepproximately equal to an eigenvector of the covarianceatrix with eigenvalue b0− km ,b0l2. The value of b0 de-

ends on the variance of the stochastic variable along theirection of the first eigenvector. The value km ,b0l2 mea-ures the distance of the mean along the b0 direction. In-uitively we can say that the eigenvectors of the correla-ion and covariance matrices are very similar if theollowing holds: The variation along the mean directionhat is not explained by shifting the mean is larger thanhe variations along the directions perpendicular to theean. We see especially that if the mean is exactly pro-

ptm

3Ttsfivtef

scsAt

rpdsw3l

inatdnv

sWfigtfiglgbiltwh

mtnRFSava

gtttsT

Fect


ortional to the first eigenvector of the correlation matrix,hen the eigenvectors of the correlation and covarianceatrices are identical.

. EXPERIMENTShe discussion above shows that the first eigenvector ofhe correlation matrix is always nonnegative. It alsohows that under certain conditions (but not always) thisrst eigenvector is similar to the mean and the first eigen-ector of the covariance matrix. It is therefore of interesto investigate the relation between the mean and the firstigenvectors of the correlation and covariance matricesor a number of relevant databases of color spectra.

In our experiments we use the spectra from the Mun-ell atlas as one representative database of manufacturedolors, and we use the eight multichannel images de-cribed in Ref. 21 as examples of measured color spectra.s illumination sources we use Planck blackbody radia-

ors and a database of measured artificial light sources.The Munsell spectra are measured in the wavelength

ange 380–800 nm in 1-nm increments. In some of our ex-eriments we downsampled the spectra from the Munsellatabase to the range 400–760 nm in 2-nm steps since thepectra of the illumination sources in this experimentere available only in that range. The images consist of1 channels representing the interval 410–710 nm in0-nm increments.

In Fig. 1 we show the results of one of our experimentsnvolving the Munsell reflectance spectra and the illumi-ation sources. In this figure the color signals are gener-ted from Munsell reflectance spectra under the illumina-ion TLD18W35White. The figure shows the spectralistribution of the light source, the mean of the color sig-als, and the first eigenvectors of the correlation and co-ariance matrices.

In Tables 1 and 2 and Figs. 2 and 3 we describe the re-ults of similar experiments but now with measured data.e investigated the similarities between the mean, the

rst eigenvector of the correlation matrix, and the first ei-envector of the covariance matrix. In all cases we foundhat the scalar product of the normalized mean and therst eigenvector of the correlation matrix had valuesreater than 0.99. In Table 1 we list the value of the sca-ar product of the normalized mean vector and the first ei-envector of the correlation matrix for the Munsell data-ase (no illumination applied) and the multispectralmages. In the experiments in which we simulated the il-umination of the Munsell chips with Planck spectra ofemperatures 4000 K–20,000 K with 1000-K incrementse found values of the scalar product that were evenigher.We computed the first eigenvectors of the correlationatrix and the normalized mean vector for the reflec-

ance spectra in the Munsell database and the color sig-als obtained from the multispectral images described inef. 21. The results for the Munsell database is shown inig. 2. Among the eight scenes in the image database,cene 1 had the lowest similarity between the eigenvectornd the mean. The normalized mean and the first eigen-ector of the correlation matrix computed from Scene 1re shown in Fig. 3.

We also investigated the similarity between the first ei-envector of the correlation matrix and the first eigenvec-or of the covariance matrix. As expected, it turns out thathey are very similar, for both the Munsell database andhe multichannel images. The numerical values of thecalar products of these two eigenvectors are listed inable 2.

Table 1. Scalar Product of the Normalized Meanand the First Eigenvector of the Correlation

Matrix for the Munsell Chips and MultichannelImages of Natural Scenes

Name Scalar Product

Munsell 0.9996Scene 1 0.9879Scene 2 0.9996Scene 3 0.9999Scene 4 0.9996Scene 5 0.9987Scene 6 0.9986Scene 7 0.9981Scene 8 0.9991

Table 2. Scalar Product of the First Eigenvectorsof the Correlation and Covariance Matrices for

the Munsell Chips and Multichannel Images

Name Scalar Product

Munsell 0.9966Scene 1 0.9951Scene 2 0.9959Scene 3 0.9975Scene 4 0.9800Scene 5 0.9820Scene 6 0.9929Scene 7 0.9927Scene 8 0.9966

ig. 1. Illumination spectrum, mean of color signals, and firstigenvectors of the correlation and covariance matrices, for theolor signals derived from the Munsell chips under the illumina-ion TLD18W35 White.

sammaav

egieuradgT

4WRnlttTtpltthmwivtsf

Fp

Fp

Fcm


These similarities hold not only for light sources withmooth distributions but also for artificial light sources,s illustrated in Table 3. In this experiment we used theeasured reflectance spectra from the Munsell atlas,ultiplied them by the spectral distribution of the lamp,

nd computed the scalar product of the normalized meannd the first eigenvector of the correlation matrix. Thesealues are listed in the table.

In all experiments we computed the mean and the firstigenvector. We estimated the dominance of the first ei-envalue with the formula given in Eq. (14) and comparedt with the ratio of the first eigenvalue and the trace. Anxample is shown in Fig. 4, where 3’s represent the val-es of the first eigenvalue divided by the trace and circlesepresent the estimated values. The similarity holds forll illumination sources and for the Munsell spectralatabase itself, i.e., the case where the illumination isiven by the vector that has the value one everywhere.his is shown as illumination number 16 in Fig. 4.

ig. 2. Mean and first eigenvector of the correlation matrix com-uted from the reflectance spectra in the Munsell database.

ig. 3. Mean and first eigenvector of the correlation matrix com-uted from the color signals in the multispectral image: Scene 1.

. DISCUSSION AND CONCLUSIONSe showed first that the Perron–Frobenius (Krein–utman) theory implies that for stochastic processes ofonnegative signals the eigenvector (eigenfunction) be-

onging to the largest eigenvalue of the correlation opera-or has only nonnegative values. Under certain conditionshis eigenvector (eigenfunction) is positive everywhere.hen we concluded that for many processes (including allhe color spectral databases investigated by us so far) thisroperty implies that the mean of the process is very simi-ar to this first eigenvector. We related this mean value tohe value of the first eigenvalue in comparison with therace of the correlation matrix. This gives a measure ofow dominant this eigenvalue is. We showed that thiseasure can be estimated from the correlation matrixithout computing the eigenvalues themselves. We also

nvestigated the conditions under which the first eigen-ectors of the correlation and covariance matrices are ofhe same form. We investigated a database of reflectancepectra from the Munsell color atlas (combined with dif-erent illumination sources) and spectral distributions

Table 3. Scalar Product of the First Eigenvectorsof the Correlation and the Normalized Mean

Vector for the Munsell Chips and IlluminationSources

No. Name Scalar Product

1 TLD18W18Blue 0.999932 TLD18W15Red 1.000003 TLD18W17Green 0.999914 F18W840Coolwhite 0.999765 TLD18W16Yellow 0.999946 TLD18W965DeLuxe 0.999917 TLD18W827ExtraWarmWhite 0.999818 TLD18W35White 0.999609 TLD18W835White 0.99982

10 TLD18W840CoolWhite 0.9997611 TLD18W860PolyluxXL 0.9996612 DeLuxeNatural 0.9998213 Tungsten60W 0.9997614 Tungsten100W 0.9997115 Halogen75W 0.99968

ig. 4. Ratio of first eigenvalue and trace and its estimation:olor signals generated by Munsell spectra under different illu-inations; see Table 3 for the corresponding illumination names.

ftmw

tago(lncfifvsstp

pgc1aektwevspduionpvp

AFACBpYfistiofiAT

s

Ifeae

ATsmnTnfatSavpm

t

R


rom multichannel images and showed that for those da-abases the mean, the first eigenvector of the correlationatrix, and the first eigenvector of the covariance matrixere very similar.Finally, we remark that this complex relation between

he mean vector, the first eigenvectors of the correlationnd covariance matrices and the dominance of the first ei-envalue has some interesting consequences for adaptiver learning systems. In the literature on neural networksand numerical mathematics) it has been known for aong time (see Ref. 22 for an early reference) that neuraletworks can compute eigenvectors from examples. Espe-ially in the case of stochastic processes with dominantrst eigenvalue, the learning process is comparativelyast. Since the processes studied in this paper all haveery dominant first eigenvalues it can be expected thatuch learning processes converge very fast to the requiredolution when applied to these data sets. An illustrationhat this is really the case is shown in the following ex-eriment.In the experiment we generated color signals by multi-

lying the Munsell spectra with the illumination sourceiven by the Planck blackbody radiator of 4000 K. Eacholor signal was defined in the range 380–800 nm with-nm sampling, i.e., a vector of size 421. At the beginningweight vector w1 of unit length with random (positive)

ntries was generated. Then we selected at each iterationa random spectrum s from the database and computed

he result v= ks ,wls and generated the new weight vectork+1=v / ivi. Each iteration thus consists of a matching op-

ration j= ks ,wl, a weighting v=js, and a normalization/ ivi. Since the first eigenvalue is dominating, it can behown that this iteration leads to an amplification of theroportion of the vector that points in the first eigenvectorirection. The result of one such experiment where wesed 10,000 iterations with a random color signal at each

teration is shown in Fig. 5. We see that the weight vectorbtained is very similar to the illumination source and theormalized mean. This is only an illustration of the basicroperties of such an adaptation process to show thatery simple adaptation rules are able to extract importantroperties from a database of color signals.

Fig. 5. Learning the eigenvector from the database.

PPENDIX A: BASIC FACTS FROMUNCTIONAL ANALYSISBanach space is a vector space with a norm in which allauchy sequences have a limit point. A Hilbert space is aanach space in which the norm is defined by a scalarroduct. A linear operator between Banach spaces X andis compact if for each bounded sequence xn[X we can

nd a subsequence xk[X such that Txk is a convergentequence in Y. For a bounded linear operator T the spec-ral radius rsTd is given by rsTd=limn→`iTni1/n, where iTis the norm of the operator. If T :X→Y is a linear continu-us operator, then the adjoint operator T* :Y→X is de-ned by the equation kTx ,yl= kx ,T*yl for all x[X ,y[Y.n operator T is Hermitian or self-adjoint if T :X→X and=T*.A special case of the general framework is given by the

tandard L2 spaces and kernel operators, where

sTfdsyd =E Ksx,ydfsxddx.

n our application the kernel function K is the correlationunction, and we assume that it is square integrable; i.e.,uKsx ,ydu2dxdy,`. The correlation function is symmetricnd real „Ksx ,yd=Ksx ,yd=Ksy ,xd…, and therefore the op-rator T is self-adjoint.

CKNOWLEDGMENTShe Munsell database23 and the database of illuminationpectra was provided by the Color Group at the Depart-ent of Computer Science, Joensuu University of Tech-ology, Joensuu, Finland (http://cs.joensuu.fi/~spectral).he multichannel images are described in Ref. 21. The fi-ancial support of the Swedish Research Council is grate-

ully acknowledged. We also thank a reviewer for a valu-ble comment on an earlier version of this paper. Part ofhis work was done while R. Lenz was at the Biomedicalensing and Imaging Group, Institute Human Sciencend Biomedical Engineering, National Institute of Ad-anced Industrial Science and Technology, Tsukuba, Ja-an on a scholarship from the Japan Society for the Pro-otion of Science.

The authors may be reached by e-mail: {reile,habu}@itn.liu.se.

EFERENCES1. H. B. Barlow, “The coding of sensory messages,” in Current

Problems in Animal Behaviour, W. H. Thorpe and O. L.Zangwill, eds. (Cambridge U. Press, Cambridge, UK, 1961)pp. 331–360.

2. D. J. Field, “Relations between the statistics of naturalimages and the response properties of cortical cells,” J. Opt.Soc. Am. A 4, 2379–2394 (1987).

3. J. J. Atick and N. Redlich, “What does the retina knowabout natural scenes,” Neural Comput. 4, 449–572 (1992).

4. C. A. Párraga, G. Brelstaff, T. Troscianko, and I. R.Moorehead, “Color and luminance information in naturalscenes,” J. Opt. Soc. Am. A 15, 563–569 (1998).

5. P. Reinagel and S. Laughlin, “Natural stimulus statistics,”Network Comput. Neural Syst. 12, 237–240 (2001).

1

1

1

1

1

1

1

1

1

1

2

2

2

2


6. A. Levy and J. Rubinstein, “Hilbert-space Karhunen–Loevetransform with application to image analysis,” J. Opt. Soc.Am. A 16, 28–35 (1999).

7. F. R. Gantmacher, Matrizentheorie (Springer-Verlag, Berlin1986).

8. N. Dunford and J. T. Schwartz, Part III, Spectral Operators(Interscience, New York, 1988).

9. J. Toland, “Self-adjoint operators and cones,” J. LondonMath. Soc. Ser. 2 53, 167–183 (1996).

0. R. D. Nussbaum, “Eigenvectors of order-preserving linearoperators,” J. London Math. Soc. Ser. 2 58, 480–496 (1998).

1. K. Yosida, Functional Analysis (Springer-Verlag, Berlin,1978).

2. S. Mallat, A Wavelet Tour of Signal Processing (Academic,New York, 1999).

3. R. Lenz, “Estimation of illumination characteristics,” IEEETrans. Image Process. 10, 1031–1038 (2001).

4. R. Lenz, “Two stage principal component analysis of color,”IEEE Trans. Image Process. 11, 630–635 (2002).

5. H. Yilmaz, On Color Perception, Vol. XX of InternationalSchool of Physics, Enrico Fermi (Academic, New York,

1962), pp. 239–251.6. H. L. Resnikoff, “Differential geometry and color

perception,” J. Math. Biol. 1, 97–131 (1974).7. M. Brill and G. West, “Group theory of chromatic

adaptation,” Farbe 31, 4–22 (1983/1984).8. H. Fairman and M. Brill, “The principal components of

reflectances,” Color Res. Appl. 29, 104–110 (2004).9. J. Worthey and M. Brill, “Principal components applied to

modeling: Dealing with the mean vector,” Color Res. Appl.29, 261–266 (2004).

0. M. Brill, “A non-PC look at principal components,” ColorRes. Appl. 28, 69–71 (2003).

1. S. Nascimento, F. Ferreira, and D. Foster, “Statistics ofspatial cone-excitation ratios in natural scenes,” J. Opt.Soc. Am. A 19, 1484–1490 (2002).

2. E. Oja, “A simplified neuron model as a principlecomponent analyser,” J. Math. Biol. 15, 267–273 (1982).

3. J. Parkkinen, J. Hallikainen, and T. Jaaskelainen,“Characteristic spectra of Munsell colors,” J. Opt. Soc. Am.A 6, 318–322 (1989).

statistical properties of color-signal spaces

Documents