an equivalent illuminant model for the effect of surface ... · an equivalent illuminant model for...

DRAFT – Do Not Quote – Comments Welcome http://journalofvision.org/ 1

DOI Draft of March, 2004 ISSN © 2002 ARVO

An Equivalent Illuminant Model for the Effect of SurfaceSlant on Perceived Lightness

M. BlojDepartment of Optometry, University of Bradford

Bradford, UK

C. RipamontiDepartment of Psychology, University of Pennsylvania Philadelphia, PA,

USA

K. MithaDepartment of Psychology, University of Pennsylvania Philadelphia, PA,

USA

R. HauckDepartment of Psychology, University of Pennsylvania Philadelphia, PA,

USA

S. GreenwaldDepartment of Psychology, University of Pennsylvania Philadelphia, PA,

USA

D. H. BrainardDepartment of Psychology, University of Pennsylvania Philadelphia, PA,

USA

Keywords: computational vision, equivalent illuminant, lightness constancy, scene geometry, surface slant

AbstractIn the companion paper (Ripamonti

et al., 2004), we present data thatmeasure the effect of surface slant onperceived lightness. Observers areneither perfectly lightness constantnor luminance matchers, and there isconsiderable individual variation inperformance. This paper develops aparametric model that accounts for howeach observer’s lightness matches varyas a function of surface slant. Themodel is derived from consideration ofan inverse optics calculation thatcould achieve constancy. The inverseoptics calculation begins withparameters that describe theillumination geometry. If theseparameters match those of the physicalscene, the calculation achievesconstancy. Deviations in the model’sparameters from those of the scenepredict deviations from constancy. Weused numerical search to fit the modelto each observers data. The modelaccounts for the diverse range ofresults seen in the experimental datain a unified manner, and examinationof its parameters allows

interpretation of the data that goesbeyond what is possible with the rawdata alone. In particular, the modelallows calculation of a constancyindex that takes on a value of 0 forluminance matching and 1 for perfectconstancy. Across our experiments,the average constancy index was 0.57.

IntroductionIn the companion paper (Ripamonti

et al., 2004), we presentpsychophysical data that measure howperceived surface lightness varieswith scene geometry. In particular,we measured the effect of surfaceslant. The data indicate thatobservers take geometry into accountwhen they judge surface lightness, butthat there are large individualdifferences. This paper develops aquantitative model of our data. Themodel is derived from an analysis ofthe physics of image formation and ofthe computations that the visualsystem would have to perform toachieve lightness constancy. Themodel allows for failures of lightnessconstancy by supposing that observersdo not perfectly estimate the lighting

Bloj et al. 2

geometry. Individual variation isaccounted for within the model byparameters that describe eachobserver’s representation of thatgeometry.

Figure 1 replots experimental datafor three observers (HWK, EEP, andFGS) from Ripamonti et al. (2004).Observers matched the lightness of astandard object to a palette oflightness samples, as a function ofthe slant of the standard object. Thedata consist of the normalizedrelative match reflectance at eachslant. If the observer had beenperfectly lightness constant, the datawould fall along a horizontal line,indicated in the plot by the reddashed line. If the observer weremaking matches by equating thereflected luminance from the standardand palette sample, the data wouldfall along the blue dashed curvesshown in the figure. The completedata set demonstrates reliableindividual differences ranging fromluminance matches (e.g. HWK) toapproximations of constancy (e.g.FGS). Most of the observers, though,showed intermediate performance (e.g.EEP).

Figure 1 about here

Figure 1. Normalized relative matches, replotted fromRipamonti et al. (2004). Data are for Observer HWK (Paintinstructions), Observer EEP (Neutral instructions), andObserver FGS (Neutral instructions). See companion paper forexperimental details.

Given that observers are neitherperfectly lightness constant norluminance matchers, our goal is todevelop a parametric model that canaccount for how each observer’smatches vary as a function of slant.Establishing such a model offersseveral advantages. First, individualvariability may be interpreted interms of variation in modelparameters, rather than in terms ofthe raw data. Second, once aparametric model is established, onecan study how variations in the sceneaffects the model parameters (c.f.Krantz, 1968; Brainard & Wandell,1992)). Ultimately, the goal is todevelop a theory that allows

prediction of lightness matches acrossa wide range of scene geometries.

A number of broad approaches havebeen used to guide the formulation ofquantitative models of contexteffects. Helmholtz (Helmholtz, 1896)suggested that perception should beconceived of as a constructedrepresentation of physical reality,with the goal of the constructionbeing to produce stablerepresentations of object properties.The modern instantiation of this ideais often referred to as thecomputational approach tounderstanding vision (Marr, 1982;Landy & Movshon, 1991). Under thisview, perception is difficult becausemultiple scene configurations can leadto the same retinal image. In thecase of lightness constancy, theambiguity arises because illuminantintensity and surface reflectance cantrade off to leave the intensity ofreflected light unchanged.

Given the ambiguity of the retinalimage about the physical configurationof the scene, what we see must dependnot only on the image but also on therules the visual system employs tochoose one configuration as thatperceived. Although this generalnotion is well accepted, differentauthors choose to formulate the natureof these rules in a variety of ways,with some focusing on constraintsimposed by known mechanisms (e.g.Stiles, 1967; Cornsweet, 1970) andothers on constraints imposed by thestatistical structure of theenvironment (e.g. Gregory, 1968; Marr,1982; Landy & Movshon, 1991; Wandell,1995; Geisler & Kersten, 2002; Purves& Lotto, 2003). In our view, themodeling agenda is less to decideamongst these broad conceptions butrather to adopt one of them and use itto provide a quantitative account of awide range of data. Here we adopt thebasic computational approach and useit to develop a model of our data.Our model is essentially identical tothat formulated recently by Boyaci etal. (2003) to account for theirmeasurements of the effect of surfaceslant on perceived lightness.

Bloj et al. 3

The starting point is the idea isthat the visual system attempts toparse the retinal image and recoverthe physical characteristics of thescene, so that perceived lightnesswould be closely correlated withobject surface reflectance. In thisapproach, lightness constancy is foundwhen the visual system is able to dothis accurately, while deviations fromconstancy indicate inaccuracies in thecalculation. In previous work, wehave elaborated equivalent illuminantmodels of observer performance fortasks where surface mode or surfacecolor was judged (Speigle & Brainard,1996; Brainard, Brunt, & Speigle,1997; see also Brainard, Wandell, &Chichilnisky, 1993; Maloney & Yang,2001; Boyaci et al., 2003). In suchmodels, the observer is assumed to becorrectly performing a constancycomputation, with the one exceptionthat their estimate of the illuminantdeviates from the actual illuminant.Thus in these models, theparameterization of the observer’silluminant estimate determines therange of performance that may beexplained, with the detailedcalculation then following from ananalysis of the physics of imageformation.

Equivalent Illuminant Model

OverviewOur model is derived fromconsideration of an inverse opticscalculation that could achieveconstancy. The inverse opticscalculation begins with parametersthat describe the illuminationgeometry. If these parameters matchthose of the physical scene, thecalculation achieves constancy.Deviations in the model’s parametersfrom those of the scene predictdeviations from constancy. In the nextsections we describe the physicalmodel of illumination and how thismodel can be incorporated into aninverse optics calculation to achieveconstancy. We then show how theformal development leads to a

parametric model of observerperformance.

Physical ModelConsider a Lambertian flat matte

standard object1 that is illuminatedby a point2 directional light source.The standard object is oriented at aslant N with respect to a referenceaxis (x-axis in Figure 2). The lightsource is located at a distance dfrom the standard surface. The lightsource azimuth is indicated by D andthe light source declination (withrespect to the z-axis) by D.

The luminance Li, Nof the light

reflected from the standard surface idepends on its surface reflectance ri,its slant N, and the intensity of theincident light E:

Li, N= riE . (1)

When the light arrives only directlyfrom the source, we can write

E = ED (2)

where

ED =

ID sin D[cos( D − N )]

d2 . (3)

Here ID represents the luminousintensity of the light source.Equation 3 applies when−90° ≤ ( D − N ) ≤ 90°. For a purelydirectional source and ( D − N ) outsideof this range, ED = 0.

Figure 2 about here

Figure 2. Reference system centered on the standard object.The standard object is oriented so that its surface normal formsan angle N with respect to the x-axis. The light source islocated at a distance d from this point, the light sourceazimuth (with respect to the x-axis) is D , and the light sourcedeclination (with respect to the z-axis) is D .

In real scenes, light from a sourcearrives both directly and afterreflection off other objects. Forthis reason, the incident light E canbe described more accurately as acompound quantity made of thecontribution of directional light ED

and some diffuse light EA. The termEA provides an approximate descriptionof the light reflected off other

Bloj et al. 4

objects in the scene. We rewriteEquation 2 as

E = ED + EA (4)

and Equation 1 becomes

Li, N

= riID sin D[cos( D − N )]

d2 + EA

. (5)

The luminance of the standard surfaceLi, N

reaches its maximum value when

D − N = 0 and its minimum for−90° ≤ ( D − N ) ≤ 90°. In the latter caseonly the ambient light EA illuminatesthe standard surface.

It is thus useful to simplifyEquation 5 by factoring out amultiplicative scale factor that isindependent of N:

Li, N= (cos( D − N ) + FA ) . (6)

In this expression = ri

ID sin D

d 2 and FA

is given by FA =

d2 EA

ID sin D.

Physical Model FitHow well does the physical model

describe the illumination in ourapparatus.? We measured the luminanceof our standard objects under allexperimental slants, and averagedthese over standard objectreflectance. Figure 3 (solid circles)shows the resulting luminances fromeach experiment of the companion paper(Ripamonti et al., 2004) plottedversus the standard object slant. Foreach experiment, the measurements arenormalized to a value of 1 at N = 0°.We denote the normalized luminances by

LN

norm. The solid curves in Figure 3denote the best fit of Equation 6 tothe measurements, where D, FA and were treated as a free parameters andchosen to minimize the mean squarederror between model predictions andmeasured normalized luminances.

Figure 3 about here

Figure 3. The green symbols represent the relative normalisedluminance measured for standard objects used in (Ripamontiet al., 2004) and the colored curves illustrate the fit of themodel described in the text. Top panel corresponds to the lightsource set-up used in Experiments 1 and 2, middle panel toExperiment 3 light source on the left, bottom panel forExperiment 3 light source on the right.

The fitting procedure returns twoestimated parameters of interest: theazimuth D of the light source and theamount FA of ambient illumination.(The scalar simply normalizes thepredictions in accordance with thenormalization of the measurements.)We can represent these parameters in apolar plot, as shown in Figure 4. Theazimuthal position of the plottedpoints represents D, while the radiusv at which the points are plotted isa function of FA:

v =

1FA + 1

. (7)

If the light incident on the standardis entirely directional then theradius of the plotted point will be 1.In the case where the light incidentis entirely ambient, the radius willbe 0.

Figure 4 about here

Figure 4. Light source position estimates of the physical model.Green lines represent the light source azimuth as measured inthe apparatus. In Experiments 1, 2 and 3 -light source on theleft- the actual azimuth was D = -36°. In Experiment 3 -lightsource on the right- the actual azimuth was D = 23°. The redsymbol represents light source azimuth estimated by the modelfor Experiments 1 and 2 ( D = -25°). For the light source onthe left, in Experiment 3, the model estimate is indicated inblue ( D = -30°); for the light source on the right in purple( D = 25°). The radius of the plotted points providesinformation about the relative contributions of directional andambient illumination to the light incident on the standard objectthrough Equation 7. The radius of the outer circle in the plot is1. The parameter values obtained for FA are FA =0.18(Experiments 1 and 2), FA =0.43 (Experiment 3 left), andFA =0.43 (Experiment 3 right).

The physical model provides a goodfit to the dependence of the measuredluminances on standard object slant.It should be noted, however, that therecovered azimuth of the directionallight source differs from our directmeasurement of this azimuth. The most

Bloj et al. 5

likely source of this discrepancy isthat the ambient light arising fromreflections off the chamber walls hassome directional dependence. Thisdependence is absorbed into themodel’s estimate of D.

Equivalent Illuminant ModelSuppose an observer has full

knowledge of the illumination andscene geometry and wishes to estimatethe reflectance of the standardsurface from its luminance. FromEquation 6 we obtain the estimate

˜ r i, N=

Li, N

(cos( D − N )+ FA ). (8)

We use a tilde to denote perceptualanalogs of physical quantities.

To the extent that the physicalmodel accurately predicts theluminance of the reflected light,Equation 8 predicts that theobserver’s estimates of reflectancewill be correct and thus Equation 8predicts lightness constancy. Toelaborate Equation 8 into a parametricmodel that allows failures ofconstancy, we replace the parametersthat describe the illuminant withperceptual estimates of theseparameters:

˜ r i, N=

Li, N

(cos( ˜ D − N )+ ˜ F A )

(9)

where ˜ D and ˜ F A are perceptual analogs

of D and FA. Note that thedependence of ˜ r i, N

on slant inEquation 9 is independent of ri.

Equation 9 predicts an observer’sreflectance estimates as a function ofsurface slant, given the parameters ˜

D

and ˜ F A of the observer’s equivalentilluminant. These parameters describethe illuminant configuration that theobserver uses in his or her inverseoptics computation.

Our data analysis procedureaggregates observer matches overstandard object reflectance to producerelative normalized matches r

N

norm.The relative normalized matchesdescribe the overall dependence of

observer matches on slant. To linkEquation 8 with the data, we assumethat the normalized relative matchesobtained in our experiment (seeAppendix of Ripamonti et al., 2004)are proportional to the computed ˜ r i, N

,leading to the model prediction

r N

norm =L

N

norm

(cos( ˜ D − N )+ ˜ F A )

(10)

where is a constant ofproportionality that is determined aspart of the model fitting procedure.In Equation 10 we have substituted

LN

norm for Li, N since the contribution of

surface reflectance ri can be absorbedinto .

Equation 10 provides a parametricdescription of how our measurements ofperceived lightness should depend onslant. By fitting the model to themeasured data, we can evaluate howwell the model is able to describeperformance, and whether it cancapture the individual differences weobserve. In fitting the model, thetwo parameters of interest are ˜

D and˜ F A, while the parameter simplyaccounts for the normalization of thedata.

In generating the modelpredictions, values for N and L

N

norm

are taken as veridical physicalvalues. It would be possible todevelop a model where these were alsotreated as perceptual quantities andthus fit to the data. Withoutconstraints on how ˜

N and ˜ L N

norm arerelated to their physicalcounterparts, however, allowing theseas parameters would lead to excessivedegrees of freedom in the model. Inour slant matching experiment,observer’s perception of slant wasclose to veridical and thus using thephysical values of N seems justified.We do not have independentmeasurements of how the visual systemregisters luminance.

Bloj et al. 6

Model Fit

Fitting the modelFor each observer, we used

numerical search to fit the model tothe data. The search procedure foundthe equivalent illuminant parameters˜

D and ˜ F A and the scaling parameter that provided the best fit to thedata. The best fit was determined asfollows. For each of the threesessions k = 1,2,3 we found thenormalized relative matches for thatsession, r k, N

norm. We then found theparameters that minimized the meansquared error between the model’sprediction and these r k, N

norm. Thereason for computing the individualsession matches and fitting to these,rather than fitting directly to theaggregate r

N

norm is that the formerprocedure allows us to compare themodel’s fit to that obtained byfitting the session data at each slantto its own mean.

Model FitModel fit results are illustrated

in the left hand columns of Figures 5to 10. The dot symbols (green forNeutral instructions and purple forPaint Instructions) are observers’normalized relative matches and theorange curve in each panel shows thebest fit of our model. We also showthe predictions for luminance andconstancy matches as, respectively, ablue or red dashed line. The righthand columns of Figures 5 to 10 showthe model’s ˜

D and ˜ F A for eachobserver, using the same polar formatintroduced in Figure 4.

Figure 5 about here

Figure 5. Model fit to observers’ relative normalized matches.In the left column the green dots represent observers’ relativenormalized matches as a function of slant for Experiment 1.Error bars indicate 90% confidence intervals. The orange curveis the model’s best fit for that observer. The blue dashed curverepresents predictions for luminance matches and the reddashed line for constancy matches. The right column showsthe equivalent illuminant parameters (green symbols) in thesame polar format introduced in Figure 4. The polar plot alsoshows the illuminant parameters obtained by fitting thephysical model to the measured luminances (red symbols).The numbers at the top left of each square are the constancyindex (CI) for the observer. Observers are listed in increasingorder of CI.

Figure 6 about here

Figure 6. Model fit to observers’ relative normalized matchesfor Experiment 2. Same format as Figure 5, except thatobserver data and parameters are shown in purple.

Figure 7 about here

Figure 7. Model fit to observers’ relative normalized matchesfor Experiment 3 (light on the left, Neutral instructions). Sameformat as Figure 5.

Figure 8 about here

Figure 8. Model fit to observers’ relative normalized matchesfor Experiment 3 (light on the right, Neutral instructions). Sameformat as Figure 5. Note that the observer order differs fromFigure 7 as the ordering of the observers in terms of CI wasnot the same for left and right light positions.

Figure 9 about here

Figure 9. Model fit to observers’ relative normalized matchesfor Experiment 3 (light on the left, Paint instructions). Sameformat as Figure 6.

Figure 10 about here

Figure 10. Model fit to observers’ relative normalized matchesfor Experiment 3 (light on the right, Paint instructions). Sameformat as Figure 6. Note that the observer order differs fromFigure 9 as the ordering of the observers in terms of CI wasnot the same for left and right light positions.

With only a few exceptions, theequivalent illuminant model capturesthe wide range of performanceexhibited by individual observers inour experiment. To evaluate thequality of the fit, we can compare themean squared error for the equivalentilluminant model to the variability inthe data. To make this comparison, wefit the r k, N

norm at each session and

Bloj et al. 7

slant by their own means. Thisprovides a lower bound on the squarederror that could be obtained by anymodel. For comparison, we alsocompute the error obtained by fittingthe r k, N

norm with the luminance matchingand constancy matching predictions.Figure 11 shows the fit errors(averaged over all experiments andobservers) when the data are fit bytheir own mean, by the equivalentilluminant model, and by thepredictions of luminance and constancymatching. We see that the error fromthe equivalent illuminant model isonly slightly greater than the errorbased on predicting with the mean, andconsiderably smaller than the errorobtained with either luminancematching or constancy predictions.


Figure 11. Mean squared errors obtained when the matchingdata for each slant, session and observer is fitted by: a) its ownmean b) the equivalent illuminant model, c) the luminancematch prediction, and d) the lightness constancy prediction.

Using the ModelThe equivalent illuminant allows

interpretation of the large individualdifferences observed in ourexperiments. In the context of themodel, these differences are revealedas variation in the equivalentilluminant model parameters ˜

D and ˜ F A,rather than as a qualitativedifference in the manner in whichobservers perform the matching task.In the polar plots we see that foreach condition, the equivalentilluminant model parameters are in thevicinity of the corresponding physicalmodel parameters, but with somescatter. Observers whose dataresembles luminance matching haveparameters that plot close to theorigin, while those whose dataresemble constancy matching haveparameters that plot close to those ofthe physical model. Given the generaldifficulty of inverse optics problems,it is perhaps not surprising thatindividual observers would vary inthis regard.

Various patterns in the data shownby many observers, particularly thesharp drop in match for N = 60° whenthe light is on the left and the non-monotonic nature of the matches withincreasing slant, require no specialexplanation in the context of theequivalent illuminant model. Both ofthese patterns are predicted by themodel for reasonable values of theparameters. Indeed, striking to uswas the richness of the model’spredictions for relatively smallchanges in parameter values.

A question of interest inExperiment 3 was whether observers aresensitive to the actual position ofthe light source. That they are isindicated by comparing the parameter˜

D across changes in the physicalposition of the light source. Theaverage value of ˜

D when the lightsource was on the left in Experiment 3was -35°, compared to 16° when it wason the right. The shift in equivalentilluminant azimuth of 51° iscomparable to the corresponding shiftin the physical model parameter (55°),indicating that observers’ demonstrateconsiderable sensitivity to the lightsource position.

Constancy IndexIn previous studies, it has proved

useful to develop a constancy index(CI) that provides a rough measure ofthe degree of constancy indicated bythe data (e.g. Brainard & Wandell,1991; Arend, Reeves, Schirillo, &Goldstein, 1991; Brainard et al.,1997; Brainard, 1998; Kraft &Brainard, 1999; Delahunt & Brainard,2004). Such an abstraction from thedata often enables examination ofbroad patterns that are difficult toexamine in the raw data. We use ourmodel fit to develop such an index forour data. Let the vector

v =vsin D

vcos D

(12)

be a function of the physical model’sparameters D and FA, with the scalarv computed from FA using Equation 7

Bloj et al. 8

above. Let the vector ˜ v be theanalogous vector computed from theequivalent illuminant model parameters˜

D and ˜ F A. Then we define ourconstancy index as

C I = 1-

v − ˜ v

v . (13)

Intuitively, this index takes on avalue of 1 when the equivalentilluminant model parameters match thephysical model parameters and a valuenear 0 when the equivalent illuminantmodel parameter ˜ F A is very large.This latter case corresponds to wherethe model predicts luminance matching.

We have computed this CI for eachobserver/condition, and the resultingvalues are indicated on the top leftof each polar plot in Figures 5-10.Generally, the indices are in accordwith a visual assessment of where thedata lie with respect to the luminancematching and constancy predictions,although there are some exceptions.This is not surprising given that asingle number cannot describe allaspects of the data.

Figure 12 summarizes the constancyindices across all of our experiments.The mean constancy index was 0.57,with large individual variation. Thismean CI quantifies, in a rough manner,the overall degree of lightnessconstancy shown by our observers.Given the computational difficulty ofrecovering lighting geometry fromimages, we regard this degree ofconstancy as a fairly impressiveachievement. Across all experiments,the mean CI for observers givenNeutral Instructions (0.56) wasessentially identical to that forobservers given Paint Instructions(0.58), consistent with our previousconclusion that our instructionalmanipulation had little effect.

Figure 12 about here.

Figure 12. Constancy indices. Individual observer and meanconstancy indices are plotted for each experimental condition.Orange filled circles, individual observer indices. Green filledcircles, mean constancy indices.

Figure 13 plots the constancy indexfor each observer when the lightsource was on the right against thecorresponding index when the light wason the left. These are only moderatelycorrelated (r = 0.37). The smallpositive correlation indicates thatwhatever factors mediate the degree ofconstancy shown by individualobservers have something in commonacross the change in light sourceposition, but that this commonality isnot dominant.


Figure 13. Comparison of CI for Experiment 3. CI for theilluminant on the right side condition versus illuminant on theleft. The correlation between CIs on the left and right was0.37.

Discussion

Equivalent Illuminant ModelsThe equivalent illuminant model

presented here accounts well for thedata reported in the companion paper.The model was derived from an analysisof the computational problem oflightness constancy. The model hastwo equivalent illuminant parameters,˜

D and ˜ F A, that describe the lightinggeometry. These parameters are not,however, set by measurements of thephysical lighting geometry but are fitto each observer’s data. Given theequivalent illuminant parameters, themodel predicts the lightness matchesthrough an inverse optics calculation.

The equivalent illuminant modeldeveloped here is an instance of ageneral class of models that connectperception to computations thatachieve constancy. The generalequivalent illuminant model has thefollowing form. First one writes anexplicit imaging model that describeshow the image data available to thevisual system about an object’ssurface depends on scene parametersthat describe physical properties ofthe illumination and object surface(e.g. Equation 6). The parametersthat describe the scene illuminationare taken the parameters of theobserver model (e.g. ˜

D and ˜ F A).

Bloj et al. 9

Given these parameters, the imagemodel is inverted to recover theparameters that describe the object’ssurface (e.g. Equation 10, whichobtains an estimate of objectreflectance from the reflectedluminance). Perception of theobject’s surface properties (e.g.lightness, color, glossiness) is thentaken to be a function of theestimated object surface properties.

Through different choices ofparameterization of illuminant andsurface properties, equivalentilluminant models can be developed fora wide range of psychophysicalexperiments. The most closelyrelated model in the literature isthat of Boyaci et al. (Boyaci et al.,2003), which was formulated to accountfor lightness matching data as afunction of standard object slant.This model is substantively identicalto ours. Brainard and colleagues haveused equivalent illuminant models toaccount for data on color constancy(Brainard et al., 1997) and theperception of luminosity for chromaticstimuli (Speigle & Brainard, 1996).In these applications, the illuminantparameters describe illuminantspectral power distributions, thesurface parameters describe surfacespectral reflectance functions, andthe imaging model relates reflectedspectra to cone photoreceptorisomerization rates.

It is tempting to associate theparameters ˜

D and ˜ F A with theobservers perceptual estimate of theillumination geometry. Since ourexperiments do not explicitly measurethis aspect of perception, we have noempirical basis for making theassociation. In interpreting theparameters as observer estimates ofthe illuminant, it is important tobear in mind that they are derivedfrom surface lightness matching dataand thus, at present, should betreated as illuminant estimates onlyin the context of our model of surfacelightness. It is possible that afuture explicit comparison couldtighten the link between the derivedparameters and conscious perception of

the illuminant. Prior attempts tomake such links between implicit andexplicit illumination perception,however, have not led to positiveresults (see Rutherford, 2000).

Comparison with Other ModelsA useful approach to modeling

context effects it to split theproblem into two parts (Krantz, 1968;Brainard & Wandell, 1992). The firstpart is to determine the parametersthat vary across changes in context(or across observers). The second isto determine what features of thecontext (or observers) set theparameter values and how they do so.The equivalent illuminant modelpresented here addresses only thefirst part of the general modelingproblem. We have identified twoparameters ˜

D and ˜ F A that vary withcontext (e.g. light source position)and across observer, and connectedthese parameters to the experimentalmeasurements. Our experiments andanalysis are mainly silent about thesecond part of the general problem: wedo not address the question of whatabout the experimental stimuli causesthe parameters to vary. (The oneconclusion we draw in this regard isthat our instructional manipulationhas at most a small effect on theparameters.) In thinking about otherapproaches to modeling lightnessconstancy, it is worth keeping the twopart distinction in mind.

The retinex theory of Land andMcCann (Land & McCann, 1971) addressesboth parts of the modeling problem:the modeling computations operate onthe directly on the retinal image andproduce predictions of lightness ateach image location. Thus the retinexmodel does not explicitly parameterizethe effect of context and separate theparameterization from the effect ofcontext on the parameters.

In any case, we do not see how theretinex model can account for ourdata. The manipulation of objectslant produces only small changes inthe projected retinal image but largechanges in perceived lightness. Sinceretinex does not account for the

Bloj et al. 10

three-dimensional scene geometry, itwill have difficulty predicting oureffects(see also Gilchrist, 1980;Gilchrist, Delman, & Jacobsen, 1983).

Indeed, any model that accounts forlightness in terms of the contrastbetween the standard and its surround(either global or local, computed fromthe two-dimensional image) will havedifficulty accounting for our results,since this surround is essentiallyconstant across our slantmanipulation. Others have emphasizedthe difficulties that contrast-basedtheories of lightness encounter forspatially-rich stimuli (e.g. Gilchristet al., 1999; Adelson, 1999).

Purves and Lotto (2003) formulatean “empirical” theory of perceptionand in particular of lightnessperception. Like us, they adopt theviewpoint that the visual systemattempts to produce percepts thatrepresent object properties. Ratherthan proceeding via an analytictreatment of the computational problemof constancy, however, these authorssuggest that insight about performanceis best gained through an empiricaldatabase that relates image properties(e.g. luminance) to scene properties(e.g. reflectance). Although we haveno objection to this approach inprinciple, we remain unconvinced thatit is feasible to obtain databasessufficiently large to makequantitative predictions forexperiments such as ours. Such adatabase would have to contain a largenumber of images for surfaces of manyreflectances at many poses illuminatedby light sources in many positions.The combinatorics do not seemfavorable. Interposing explicitimaging models and inverse opticscalculations between thecharacterization of natural scenestatistics and predictions of humanperformance, as we do, mitigates thecombinatoric explosion.

In recent work, both Gilchrist(1999) and Adelson (1999) haveoutlined approaches to lightnessperception that emphasize the secondpart of the general modeling problem:these authors stress the need tounderstand how the visual system

segments the image into separate“Frameworks” (Gilchrist) or“Atmospheres” (Adelson) and the needto understand what image features ineach “Framework”/”Atmosphere” affecthow the visual system transformssurface luminance to perceivedlightness. In this sense, thesetheories are broadly complementary tothe modeling approach developed here,which focuses on a quantitativecharacterization of what parametersvary as context is changed.

Extending the ModelThe challenge for the equivalent

illuminant approach we advocate hereis to pursue experiments and theoriesthat predict the equivalent illuminantto the image. An attractive featureof the equivalent illuminant approachis that the model parameters areexactly the quantities estimated bycomputer vision algorithms designed toachieve constancy. Thus to solve thesecond part of the modeling problem,any equivalent illuminant model may becoupled with any algorithm thatattempts to estimate its parametersfrom image data. We have begun toexplore this in our work on colorconstancy, where we have coupled analgorithm for spectral illuminantestimation (Brainard & Freeman, 1997)with our equivalent illuminant modelfor successive color constancy(Brainard, 1998). In initial tests,this general approach has led tosuccessful predictions across a widerange of experimental conditions(Brainard, Kraft, & Longère, 2003).As computational algorithms forestimating illumination geometrybecome available, these may be used ina similar recipe with the equivalentilluminant model presented here. Inthe meantime it seems of interest toexplore experimentally how theequivalent illuminant parameters varywith more systematic changes in thescene (e.g. light position, relativestrength of directional and ambientillumination), as these measurementsmay provide clues as to what imagefeatures affect the parameters.

Bloj et al. 11

AcknowledgementsThis work was supported NIH EY

10016. We thank B. Backus, H. Boyaci,A. Gilchrist, L. Maloney, J. Nachmias,and S. Sternberg for helpfuldiscussions.

References

Adelson, E. H. (1999). Lightnessperception and lightnessillusions. In M. Gazzaniga (Ed.),The New Cognitive Neurosciences,2nd ed. (pp. 339-351). Cambridge,MA: MIT Press.

Arend, L. E., Reeves, A., Schirillo,J., & Goldstein, R. (1991).Simultaneous color constancy:papers with diverse Munsellvalues. Journal of the OpticalSociety of America A, 8, 661-672.

Boyaci, H., Maloney, L. T., & Hersh,S. (2003). The effect of perceivedsurface orientation on perceivedsurface albedo in binocularlyviewed scenes. Journal of Vision,3, 541-553.

Brainard, D. H. (1998). Colorconstancy in the nearly naturalimage. 2. achromatic loci. Journalof the Optical Society of AmericaA, 15, 307-325.

Brainard, D. H., Brunt, W. A., &Speigle, J. M. (1997). Colorconstancy in the nearly naturalimage. 1. asymmetric matches.Journal of the Optical Society ofAmerica A, 14, 2091-2110.

Brainard, D. H., & Freeman, W. T.(1997). Bayesian color constancy.Journal of the Optical Society ofAmerica A, 14(7), 1393-1411.

Brainard, D. H., Kraft, J. M., &Longère, P. (2003). Colorconstancy: developing empiricaltests of computational models. InR. Mausfeld & D. Heyer (Eds.),Colour Perception: Mind and thePhysical World (pp. 307-334).Oxford: Oxford University Press.

Brainard, D. H., & Wandell, B. A.(1991). A bilinear model of theilluminant's effect on colorappearance. In M. S. Landy & J. A.Movshon (Eds.), ComputationalModels of Visual Processing.Cambridge, MA: MIT Press.

Brainard, D. H., & Wandell, B. A.(1992). Asymmetric color-matching:how color appearance depends onthe illuminant. Journal of theOptical Society of America A,9(9), 1433-1448.

Brainard, D. H., Wandell, B. A., &Chichilnisky, E.-J. (1993). Colorconstancy: from physics toappearance. Current Directions inPsychological Science, 2, 165-170.

Cornsweet, T. N. (1970). VisualPerception. New York: AcademicPress.

Delahunt, P. B., & Brainard, D. H.(2004). Does human color constancyincorporate the statisticalregularity of natural daylight?Journal of Vision, 4(2), 57-81.

Geisler, W. S., & Kersten, D. (2002).Illusions, perception, and Bayes.Nature Neuroscience, 5, 508-510.

Gilchrist, A., Kossyfidis, C., Bonato,F., Agostini, T., Cataliotti, J.,Li, X., Spehar, B., Annan, V., &Economou, E. (1999). An anchoringtheory of lightness perception.Psychological Review, 106, 795-834.

Gilchrist, A. L. (1980). When doesperceived lightness depend onperceived spatial arrangement?Perception and Psychophysics, 28,527-538.

Gilchrist, A. L., Delman, S., &Jacobsen, A. (1983). Theclassification and integration ofedges as critical to theperception of reflectance andillumination Perception andPsychophysics 33, 425-436.

Gregory, R. L. (1968). VisualIllusions. Scientific American.

Helmholtz, H. (1896). PhysiologicalOptics. New York: DoverPublications, Inc.

Bloj et al. 12

Kaufman, J. E., & Christensen, J. F.(Eds.). (1972). IES LightingHandbook; The Standard LightingGuide (5 ed.). New York:Illuminating Engineering Society.

Kraft, J. M., & Brainard, D. H.(1999). Mechanisms of colorconstancy under nearly naturalviewing. Proc. Nat. Acad. Sci.USA, 96(1), 307-312.

Krantz, D. (1968). A theory of contexteffects based on cross-contextmatching. Journal of MathematicalPsychology, 5, 1-48.

Land, E. H., & McCann, J. J. (1971).Lightness and retinex theory.Journal of the Optical Society ofAmerica, 61, 1-11.

Landy, M. S., & Movshon, J. A. (Eds.).(1991). Computational Models ofVisual Processing. Cambridge, MA:MIT Press.

Maloney, L. T., & Yang, J. N. (2001).The illuminant estimationhypothesis and surface colorperception. In R. Mausfeld & D.Heyer (Eds.), Colour Perception:From Light to Object. Oxford:Oxford University Press.

Marr, D. (1982). Vision. SanFrancisco: W. H. Freeman.

Purves, D., & Lotto, R. B. (2003). WhyWe See What We Do: An EmpiricalTheory of Vision. Sunderland, MA:Sinauer.

Ripamonti, C., Bloj, M., Mitha, K.,Greenwald, S., Hauck, R., Maloney,S. I., & Brainard, D. H. (2004).Measurements of the effect ofsurface slant on perceivedlightness. Journal of Vision,submitted.

Rutherford, M. D. (2000). The role ofillumination perception in colorconstancy. Unpublished Ph.D., UCSanta Barbara, Santa Barbara.

Speigle, J. M., & Brainard, D. H.(1996). Luminosity thresholds:effects of test chromaticity andambient illumination. Journal ofthe Optical Society of America A,13(3), 436-451.

Stiles, W. S. (1967). Mechanismconcepts in colour theory. Journalof the Colour Group, 11, 106-123.

Wandell, B. A. (1995). Foundations ofVision. Sunderland, MA: Sinauer.

Notes 1 A Lambertian surface is a uniformlydiffusing surface with constantluminance regardless of the directionfrom which is viewed (Wyszecki andStiles, 1982).2A light source whose distance fromthe illuminated object is at least 5times its main dimension is consideredto be a good approximation of a pointlight source (Kaufman & Christensen,1972).

Bloj et al., Figure 1

0

90 90

60 60

30 30

BST0.40

0

90 90

60 60

30 30

ADN0.43

0

90 90

60 60

30 30

CVS0.46

0

90 90

60 60

30 30

DTW0.48

0

90 90

60 60

30 30

EEP0.55

0

90 90

60 60

30 30

FGS0.69

0

90 90

60 60

30 30

GYD0.69

Model estimationPhysical estimation

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

Rel

ativ

e m

atch

ref

lect

ance

Standard object slant

Observer match

Luminance matchModel Fit

Constancy

�� Bloj et al., Figure 5

0

90 90

60 60

30 30

LEF0.32

0

90 90

60 60

30 30

IBO0.50

0

90 90

60 60

30 30

HWC0.60

0

90 90

60 60

30 30

JPL0.62

0

90 90

60 60

30 30

KIR0.75

0

90 90

60 60

30 30

NMR0.77

0

90 90

60 60

30 30

MRG0.91


60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

Rel

ativ

e m

atch

ref

lect

ance


Observer match

Luminance matchModelFit

Constancy


0

90 90

60 60

30 30

FGP0.23

0

90 90

60 60

30 30

EKS0.40

0

90 90

60 60

30 30

DDB0.41

0

90 90

60 60

30 30

ALR0.51

0

90 90

60 60

30 30

BMZ0.52

0

90 90

60 60

30 30

GPW0.55

0

90 90

60 60

30 30

CPK0.60


60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

Rel

ativ

e m

atch

ref

lect

ance


Observer match

Luminance matchModelFit

Constancy


0

90 90

60 60

30 30

BMZ0.50

0

90 90

60 60

30 30

DDB0.58

0

90 90

60 60

30 30

CPK0.67

0

90 90

60 60

30 30

FGP0.68

0

90 90

60 60

30 30

ALR0.72

0

90 90

60 60

30 30

EKS0.80

0

90 90

60 60

30 30

GPW0.82


60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

Rel

ativ

e m

atch

refle

ctan

ce


Observer matchLuminance matchModelFitConstancy


0

90 90

60 60

30 30

HWK0.31

0

90 90

60 60

30 30

JHO0.31

0

90 90

60 60

30 30

IQB0.32

0

90 90

60 60

30 30

LPS0.41

0

90 90

60 60

30 30

KVA0.49

0

90 90

60 60

30 30

MOG0.56

0

90 90

60 60

30 30

NPY0.57


60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

Rel

ativ

e m

atch

refle

ctan

ce




0

90 90

60 60

30 30

HWK0.53

0

90 90

60 60

30 30

JHO0.56

0

90 90

60 60

30 30

IQB0.66

0

90 90

60 60

30 30

LPS0.69

0

90 90

60 60

30 30

NPY0.71

0

90 90

60 60

30 30

KVA0.74

0

90 90

60 60

30 30

MOG0.76


60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

60 30 0 30 60

0.4

0.8

1.2

1.6

Rel

ativ

e m

atch

refle

ctan

ce




0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

Mean Model LuminanceMatch

ConstancyMatch

Model Type

Fit

Err

or

Bloj et al. Figure 11

0

0.2

0.4

0.6

0.8

1

0 1 2 3 4 5 6 7

Experiment

Con

stan

cy In

dex

Observers' mean

Experiment's mean

Exp

erim

ent 1

Exp

erim

ent 3

Left,

Neu

tral

Exp

erim

ent 2

Exp

erim

ent 3

Rig

ht, N

eutr

al

Exp

erim

ent 3

Left,

Pai

nt

Exp

erim

ent 3

Rig

ht, P

aint


0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

CI Left

CI

Rig

ht


an equivalent illuminant model for the effect of surface ... · an equivalent illuminant model for...

Documents