Specularity, the Zeta-image,and Information-Theoretic Illuminant

Estimation

Mark S. Drew1, Hamid Reza Vaezi Joze1, and Graham D. Finlayson2

2School of Computing Sciences,The University of East AngliaNorwich, Englandgraham@cmp.uea.ac.uk

1School of Computing ScienceSimon Fraser UniversityVancouver, BC, Canada

mark@cs.sfu.ca

2

Relative Chromaticity:

The main idea is that we can get at a good solution for the chromaticity of the light by dividing image chromaticity 3-vector by

the candidate light chromaticity e — the

“relative chromaticity” = e

[where is component-wise division].

Zeta-Image: Goal= Discover the light color (yet another!??)

3

Algorithm:

Then we show that, over pixels that are specular or white, the log of the relative chromaticity log() is perpendicular to the

light chromaticity e in color space. This

gives a useful hint for recovering e .

4

Proof; Background:

Simple image formation model --

k = R,G,B

pixelcolor

surface

light

specularity

Light is “white enough”: Borges, JOSA 1991, “Trichromatic approximation method for surface illumination” “N

eu

tral in

terf

ace

”: L

ee,

JOS

A 1

98

6,

“Meth

od f

or

com

pu

tin

g t

he s

cene-

illu

min

ant

hro

mati

city

”

5

Proof…

Now chromaticity: = {R,G,B}/{R+G+B)

Relative chrom.:

6

Proof…

…Relative chrom.:

simple!

7

Proof…

Now let’s head for a Planar Constraint:

8

Proof…

9

Therefore, let the Zeta-image† be• Definition:

where e is the chromaticity of illuminant and (x) is the chromaticity value of pixel at position x .

Properties– It has the structure of the

Kullback-Leibler Divergence from Information Theory;

– Zeta is low (near-zero) at specularities:

eexx )/)log()(

,0e 0

† Patent applied for

10

Zeta-image for illumination estimation– Planar constraint: For near-specular pixels or white

surfaces, Log-Relative-Chromaticity values are orthogonal to the light chromaticity. Or, equally, the zeta image is near zero.

– So the best light chromaticity e for an image is that

which minimizes the zeta-image for near-specular pixels (or white surfaces) => guess a domain ; then

eee

)/log(min

== Search

,

11

Search — explained:

• assume candidate e; form ; the lowest 10-percentile, say, of dot-product values could be near-specular pixels.

• Over a grid of possible light-chromaticities e,

minimize dot-product values

over candidate illuminants for those lowest 10-percentile pixels.

12

Or, Thm: Analytic solution is the geometric mean of

where is a set of bright pixels

Search is better than Analytic but slower.

13

Domain for Analytic Solution:• We start by approximating as top-5% brightness

pixels.– Could be any other method to indicate near- specular

pixels and white surface regions.

• Detecting Failure i.e., detecting images not having specularity or white surfaces in top brightness pixels:1. can stem from areas of images belonging to the brightest

surface which happens to tend to be some particular surface color.• we can simply check if these pixels are in the possible chromaticity gamut of illuminants.

2. can be a bag-of-pixels from all over the image.• we can investigate the distribution of in chromaticity space.

14

Does this work?...

divide by correct light, bottom 20% of Zeta: yes!

15

...Does this work?...

incorrect light, bottom 20% of Zeta: no!

correct light, float (inverted) Zeta

16

Details of Analytic:How did we come to the geometric mean?

Solve:

ek /

i

ie

NNN log

1/)(log

17

…Details of Analytic…

How do we know is positive?

If components were probabilities then

has structure of Kullback-Leibler Divergence:

extra bits to code samples from ek when using

codebook based on ik , so positive!

18

…Details of Analytic…

Final step: form using the geomean chromaticityfor bright pixels, and then trim to least-10% values ofZeta, and recalculate the geomean.

Does this work?

19

float (inverted) Zeta

lowzeta = bright & (zeta<quantile(zeta)),0.10);

bright

20

plots of for correct light e (O) and for

analytic solution e (x) :

21

In expts.

mask off t

he

ColorCheck

er

22

so far: Algorithm 1:Use either analytic answer, or a simple hierarchical grid search over light- chromaticity

Algorithm 2: Planar Constraint Applied as Post-Processing

Use any alg’s answer for e; take SVD of lowest-10% dot-product pixels: improves light estimate!

23

fallback position:If either of two chromaticity checks fails, use Grey-Edge.

24

Post-processing: Results.

25

Illumination Estimation: Results.

(Analytic)

26

Thanks!

Funding: Natural Sciences and Engineering Research Council of Canada