Surface Radiance Correction for Shape-from-Shading

Hossein Ragheb1 and Edwin R. Hancock2

1Department of Computer Engineering, Bu-Ali Sina University,

Hamedan, Iran, PO Box 65175-4161, ragheb@basu.ac.ir

2Department of Computer Science, University of York, York,

YO1 5DD, UK., erh@cs.york.ac.uk

Abstract

It is well known that many surfaces exhibit reflectance that is not well modelled by

Lambert’s law. This is the case not only for surfaces that are rough or shiny, but also

those that are matte and composed of materials that are particle suspensions. As a

result, standard Lambertian shape-from-shading methods can not be applied directly

to the analysis of rough and shiny surfaces. In order to overcome this difficulty, in this

paper, we consider how to reconstruct the Lambertian component for rough and shiny

surfaces when the object is illuminated in the viewing direction (retroreflection). To do

this we make use of the diffuse reflectance models described by Oren and Nayar, and

by Wolff. Our experiments with synthetic and real-world data reveal the effectiveness

of the correction method, leading to improved surface normal and height recovery.

1 Introduction

The recovery of surface shape from shading patterns has been a topic of sustained activity

in the computer vision literature for almost three decades. Some of the earliest work in

computer vision is reported in the PhD theses by Horn [1] and by Krakauer [2]. Early

developments in the field are well documented by Horn in the chapter “Height and Gradient

from Shading” which appears in the collection of papers in [3]. In addition to being limited

to surfaces of constant albedo illuminated by a parallel beam, the early work on shape-

from-shading (SFS) suffered from a number of technical problems. These include issues

of numerical instability [4], the need to deal with multiple light sources [5], and how to

initialize the surface normal directions [6]. During the 1990’s there was significant progress

in the field aimed at overcoming these problems. For instance, Kimmel and Bruckstein [7]

showed how the apparatus of level-set theory could be used to solve the differential equation

underpinning SFS, Oliensis and Dupuis [8] developed a provably correct method for surface

height recovery that involves propagation in the direction of maximum surface slope, Forsyth

and Zisserman [9] considered the problem of interreflection, Brooks et al. [10] investigated

the effects of ambiguous shading patterns, and, Kozera and Klette [4] have concentrated on

issues of numerical stability. An up-to-date review, that discusses a number of methods in

detail and which provides a comparative study is presented by Zhang et al. [11].

With hindsight, when measured against current achievements, some of the earliest work

on the subject was ambitious in its goals, allowing for both general reflectance models and

perspective geometry. Although the bulk of the recent work in SFS restricts itself to a

Lambertian model, the need for non-Lambertian reflectance models has been appreciated

from the outset. For instance, in work from photoclinometry Rindfleisch [12] has used the

non-Lambertian reflectance models of Hapke, Minnaert and Fesenkov [13, 14, 15] to recover

the slopes of lunar maria, and the reflectance map of Horn [16] is not restricted to Lambertian

reflectance. One area where non-Lambertian reflectance models are of critical importance,

and have been used with some success, is in radar SFS. This is a process of recovering

surface gradient and height from radar returns, and has proved to be an important tool

in the analysis of Megallan radar images of Venus. Here Frankot and Chellappa [17] have

used an empirical radar reflectance model in conjunction with a Fourier domain method for

reconstructing height under integrability constraints. In a recent paper, Bors et al. [18] have

developed a statistical method for radar SFS, using a digital elevation map to estimate the

radar reflectance function. However, one of the problems that can arise with radar-SFS is

the steepness of the radar beam which leads to range overlap.

1.1 Non-Lambertian Reflectance

Recently, there has been considerable interest in the computer vision literature in developing

reflectance models that can account for departures from Lambert’s law [19, 20, 21]. According

to Lambert’s law, the observed brightness at a point on the surface does not depend on the

viewing direction, and is proportional to the cosine of the incidence angle of the illuminant.

2

However, there are many situations where this simple reflectance model is violated. The

most familiar example is at the locations of specularities for shiny surfaces. Perhaps the

most widely used of these is the Phong model [22]. However, there are also situations in

which Lambert’s law is violated for non-specular reflectance. This is the case for matte

surfaces that consist of a suspension of reflecting particles in a transparent medium, those

that are rough (i.e. those in which there is a significant variation in local surface relief) and

those that are shiny (i.e. those where there is significant microfacet reflection). For rough

and shiny surfaces these effects are particularly marked at the occluding boundary where the

surface is highly inclined to the viewer direction. In the case of shiny surfaces the departure is

due to backscattering from microfacets which protrude above the surface. For rough surfaces

it is due to the foreshortening of surface cavities. For surfaces of intermediate roughness,

the diffuse scattering process is particularly complicated since it is a combination of effects

produced both by internal scattering and by the external roughness conditions. As a result

there may be limb brightening due to coherent scattering from microfacets that protrude

from the surface. The rough surface may also have cavities, and the angular distribution

of the cavity walls may also cause increases in limb brightness [23]. There are also surfaces

composed of materials that are particle suspensions in a transparent medium. These surfaces

while neither shiny, nor rough exhibit non-Lambertian reflectance that is best accounted for

by a “paint-model” [24]. Examples include semi-translucent plastics and surfaces to which

paint has been applied.

There have been many attempts to develop more complex light reflectance models for these

different types of surface. For instance, paint-models assume that the surface is composed of

a suspension of transparent refractive particles in a medium of different refractive index [2].

The Torrance-Sparrow model [25] assumes that the surface is composed of specular facets

whose orientation is governed by an angular distribution. Surface texture may be described

using a similar model in which the planar microfacets have an arbitrary reflectance function

[13, 14, 15]. Here the reflectance function could be Lambertian, and the departures from

Lambert’s law are governed by the angular distribution of the microfacets. However, an

alternative approach is to describe the surface using a roughness model based on a topo-

graphic relief distribution [26]. Although there are different definitions of surface roughness,

the parameter of the relief distribution is usually taken to be the root-mean-square height

deviation about the mean. The quantity can be obtained directly from surface-profile mea-

3

surements, or it can be calculated from scattering measurements provided that a theoretical

light scattering model is to hand. As we will see later, Oren and Nayar [23] adopt a different

measure of roughness based on the distribution of cavity wall slope angles. When estimating

the surface roughness for slightly-rough surfaces, it is standard practice to use the so called

TIS (Total Integrated Scattering) ratio measurement. However, the estimates returned by

this method are reliable only when the wavelength is long relative to the roughness. In fact

the TIS method forms the basis for a U.S. standard for effective surface roughness. As noted

by Bennett and Mattsson [26], there is no unique measure of surface roughness.

There are a number of ways in which rough surface reflectance can be modelled. For

instance, in the graphics community some purely empirical models have been developed

with the goal of developing computationally efficient tools for the purposes of performing

realistic surface synthesis [27, 28]. At the other extreme, in physics the study of rough

scattering has attracted considerable attention and has lead to the development of accurate

models using wave scattering theory and based on physically meaningful parameters that

can be directly measured [29, 30]. However, these models are not tractable for problems of

surface analysis in computer vision. As an example, the Beckmann formulation of Kirchhoff

scatter theory [29] fails to handle wide-angle scattering and large angles of incidence. The

reason for this is that Kirchhoff theory is based on a single scatter model and hence does

not account for effects such as multiple scattering or self shadowing, which occur at large

values of surface slope. Vernold and Harvey [30] have recently developed a modification of

the Beckmann model and have claimed that it gives excellent agreement with experimental

scattering data from rough surfaces at both large angles of incidence and at large scatter

angles. We have recently developed a technique for estimating the surface slope of very-

rough surfaces using this model [31]. Finally, if a model proves to be altogether too elusive,

then one approach is to acquire empirical estimates of the bidirectional reflectance function

(BRDF) from data. The BRDF is defined to be the ratio of the outgoing surface radiance to

the incoming light-source irradiance per unit solid angle. Measuring the BRDF is hence an

expensive and time consuming process since it has four degrees of freedom corresponding to

the zenith and azimuth angles of the light-source and the viewer. In a recent paper, Dana

et al. [19] have tabulated the BRDF’s of a number of different types of rough surface.

A compromise between the empirical and physics-based methods is offered by the phe-

nomenological approach adopted by Wolff [32], and, by Oren and Nayar [23] which attempts

4

to incorporate simple physics. These models attempt to account for departures from Lam-

bertian reflectance due to surface roughness or subsurface scattering in a parametrically

efficient manner. Each model modifies Lambert’s cosine law in a different way. Oren and

Nayar [23] address the problem of rough reflectance from matte surfaces. Their model ac-

counts for the angular distribution of surface cavity walls. When restricted to the case of

retroreflection (when the surface is illuminated in the viewing direction), the model adds to

Lambert’s cosine law a term that depends on the squared sine of the incidence angle. This

term results in brightening at the occluding object boundary. Wolff, on the other hand,

has a physically deeper model for diffuse reflectance from shiny but slightly rough surfaces.

The model uses an angle dependent Fresnel term to account for the refractive attenuation

of incident light at the surface-air boundary [32, 33]. This Fresnel term modifies in a multi-

plicative way the Lambertian cosine model. The effect is to depress the surface radiance for

near-normal incidence. Recently, Wolff et al. [34] have suggested a combination of these two

methodologies to model reflectance from surfaces with intermediate roughness. For the case

of retroreflection, the combined model includes both multiplication of the cosine law by a

Fresnel term, and the addition of a sine-squared term. Hence, there can be both an increase

in radiance at the limbs and a reduction in radiance at near-normal incidence.

1.2 Contribution

In this paper we are interested in whether these non-Lambertian reflectance models can

be used in conjunction with SFS. Of course the classical way in which to deal with non-

Lambertian surfaces is to follow Horn and construct a reflectance map as input to SFS

[16]. However, here we adopt a different approach, and explore whether we can use the

reflectance models to extract corrected Lambertian radiance from non-Lambertian surfaces.

This allows us to apply simple SFS algorithms, without the need to construct the reflectance

map. Although this approach allows non-Lambertian SFS to be conveniently modularized,

we do not argue that it is to be preferred to the use of the reflectance map.

The present work is also motivated by the fact that several of the most successful existing

methods for SFS are restricted in applicability to Lambertian reflectance and can not be

adapted easily to work with the reflectance map. Hence, our correction process can be used

as a simple tool to preprocess the input image data for such methods. It applies when

the viewer and light-source directions are the same. This is not an ideal configuration for

5

the perception of relief because of the lack of shadows. Several authors have investigated

different backscattering mechanisms. For instance, Orlova [35] found that the moon’s surface

reflects more light in the direction of the source (sun) than in other directions. Hapke et al.

[36] have discussed the retroreflection or opposition effect. Because the correction process

delivers corrected information about the angle of incidence at every surface location, this

allows the Lambertian SFS method to recover improved estimates of surface normal direction

and hence surface height. In other words, our correction process allows existing Lambertian

SFS methods to be applied to a greater variety of surfaces. It is also worth noting that

Lambertian correction can be useful to assist in visualization, since the limb brightening

due to roughness can result in further loss of perceived surface relief. Our experiments

with synthetic data confirm that the correction process results in improved surface normal

directions when a Lambertian SFS method is used.

The outline of this paper is as follows. Section 2 discusses the three different reflectance

models used in our analysis and explains how they may be simplified for the case of retrore-

flection. In Section 3 we describe how the simplified reflectance models outlined in Section

2 may be used to extract the corrected Lambertian component. Section 4 details experi-

ments on synthetic and real-world images. Finally, Section 5 offers conclusions and identifies

avenues for future research.

2 Diffuse Reflectance Models

Lambert’s law is the simplest model for diffuse reflectance from a surface. It has been widely

used in computer vision to recover shape from shading patterns. If θ is the angle of incidence

for the illuminant on the surface, the normalized radiance is IL(θ) = cos θ. There are a large

number of situations in which there are significant departures from Lambert’s law. This is

the case close to the occluding object boundary for all illumination directions. For illuminant

incidence angles larger than 50◦ relative to the viewer direction, there are also significant

departures from Lambert’s law both near the occluding boundary and over a large portion

of the object bounded by the shadow boundary [34]. However, it should be stressed that

the basis of Lambert’s law is purely phenomenological, and the reflectance function does

not attempt to model the physical processes of light scattering. The method does, however,

provide a surprisingly good account of reflectance from certain surfaces (packed Magnesium

6

Oxide for instance). Attempts have been made to justify the method using physics, and

these include the use of the distribution of black-body radiation from glowing particles.

In this paper we investigate whether the reflectance models described by Wolff [32] and

by Oren and Nayar [23] can be used as alternatives to Lambert’s law for rough and glossy

surfaces. In particular we investigate whether the two models can be used to perform radiance

correction with the aim of recovering the Lambertian component from the measured image

brightness. Unfortunately, since these non-Lambertian reflectance models depend on both

the incidence and reflectance angles they prove to be intractable for arbitrary geometry.

To overcome this problem, we restrict our attention to the retroreflection case where the

difference between the light-source direction ~L and the viewing direction ~V is small, i.e.

~L ≈ ~V . Although this is a restrictive assumption, it still results in methods which are of

practical utility. Few would view SFS as a tool for analyzing objects in unstructured scenes

and uncontrolled lighting. Instead, it may prove useful as a means by which to acquire

surface shape under controlled lighting conditions. The method described in this paper is

one that can be used to recover corrected Lambertian component under such conditions.

For instance, Zhang and Shah [5] have mentioned that when only one input image is used,

in order to recover the shape as completely as possible, the image has to be taken with

careful light-source placement to illuminate most of the object. Moreover, it is important

to stress that the restriction does not weaken or approximate the contributions from the

non-Lambertian component.

2.1 Oren-Nayar Model for Rough Surfaces

Oren and Nayar have developed a diffuse reflectance model for rough surfaces [23]. They

have used the roughness model proposed by Torrance and Sparrow [25] that assumes the

surface is composed of extended symmetric V-shaped cavities. Each cavity consists of two

planar facets. The width of each facet is assumed to be small compared to its length. The

roughness of the surface is specified using a probability distribution function for the facet

slopes. Further, it is assumed that the facet area is large compared to the wavelength of the

incident light, and hence, geometrical optics can be used to derive the reflectance model.

Finally, each facet is assumed to follow the Lambert’s law. In its more complex form,

the model takes into account detailed geometrical effects such as shadowing, masking and

interreflections between points on the surface. However, here we use their basic qualitative

7

reflectance model in which interreflections are ignored to simplify matters. It is interesting

to note that for the illumination conditions used in our method, both the more detailed and

the basic forms of the model are identical. According to this model (Fig. 1.a), for a point on

a surface with a roughness parameter σ, if the incident irradiance in direction (θi, φi) relative

to the surface normal is Li, then the surface radiance Lr in the reflection direction (θr, φr) is

Lr(θi, θr, φr − φi, σ) =ρ

πLi cos(θi){A + B max [0, cos(φr − φi)] sin(α) tan(β)} (1)

The parameters A and B, which are dimensionless, are only dependent on the surface

roughness σ which measures the distribution of cavity wall slope angles, and is hence

measured in degrees or radians (rd). Specifically A = 1.0 − 0.5σ2/(σ2 + 0.33) and B =

0.45σ2/(σ2 + 0.09). Also, the angles α and β are the maximum and minimum of the values

of the zenith angles θi and θr, i.e. α = max[θi, θr] and β = min[θi, θr]. It is important to

note that the model reduces to the Lambertian case when σ = 0. Finally, the parameter ρ

represents the surface albedo (reflectivity) which is assumed to be constant.

(a)0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

0

0.2

0.4

0.6

0.8

1

Incidence angle (radians)

Rad

ianc

e

0.05

0.2

0.4

0.6 0.8

1.0

(b)

Figure 1: (a) Typical geometry for the local tangent plane PN of the surface normal ~N ; (b) surface

radiance versus θi using the Oren-Nayar model (Eq. 2) for σ = 0.05, 0.2, 0.4, 0.6, 0.8, 1.0 rd.

Retroreflection: Here we use a simplified version of the Oren-Nayar model for the pur-

poses of radiance correction, i.e. the process of recovering the Lambertian component. This

method is applied to the case of retroreflection where the light-source and viewing directions

are nearly identical (~L ≈ ~V ). Under such conditions θr = θi = θ and φi = φr, and so we

can approximate the Oren-Nayar model by making the substitutions cos(φr − φi) = 1 and

α = β = θ. As a result, the simplified radiance (for ρ = 1) is

ID(θ, σ) = A cos(θ) + B sin2(θ) (2)

8

Hence, the correction to Lambert’s law is additive and proportional to sin2 θ. This term is

greatest at the occluding boundary, and hence results in limb brightening. Fig. 1.b shows

the radiance versus the incidence angle for different values of the roughness parameter σ in

Eq. (2). The effect of increasing the roughness is to make the reflectance function flatter

with incidence angle. Hence, the contrast between the limb and the remainder of the object

is reduced. It is important to note that when φi = φr, the more detailed form of the model

of Oren-Nayar appearing in [23] is identical to the basic form given by Eq. (1), since when

cos(φr − φi) = 1 effects of the interreflection term vanishes.

2.2 Wolff Model for Smooth Surfaces

Wolff has developed a physically motivated model for reflectance from smooth surfaces [32,

34]. The model accounts for subsurface refraction using a Fresnel attenuation factor, which

modifies a Lambertian reflectance function in a multiplicative way. According to this model,

if Li is the incident irradiance at angle θi relative to the surface normal through solid angle

dω, then the reflected radiance at angle θr is

Lr(θi, θr, n) = %Li cos(θi)[1 − F (θi, n)]{1 − F (sin−1[(sin θr)/n], 1/n)}dω (3)

The attenuation factor, 0 ≤ F (αi, n) ≤ 1.0, is governed by the Fresnel function

F (αi, r) =1

2

sin2(αi − αt)

sin2(αi + αt)

[

1 +cos2(αi + αt)

cos2(αi − αt)

]

(4)

where αt is the transmission angle of light into the surface which is given by Snell’s law:

r = (sin αi)/(sin αt) ⇒ αt = sin−1[(sin αi)/r] (5)

where n is the index of refraction of the dielectric medium. Using Fresnel attenuation and

Snell refraction, the model accounts for how incident light and the distribution of subsurface

scattered light behaves at a smooth air-dielectric surface boundary. The Wolff model deviates

from the Lambertian form (i.e. cos θi) when the Fresnel reflection terms become significant.

Almost all commonly found dielectric materials have an index of refraction, n, in the range

of 1.4 to 2.0. For instance, for glass or silicon n ≈ 1.5, for ceramic n ≈ 1.7 and for titanium

oxide or diamond n ≈ 2.0. As a result the Fresnel function is weakly dependent upon the

index of refraction for most dielectrics. When light is transmitted from air into a dielectric

r = n and the incidence angle is αi = θi. However, when transmission is from a dielectric

9

into air, then r = 1/n and αi = sin−1[(sin θr)/n]. Here, we do not consider the more complex

case, which can be used to model paint, in which transparent particles of one dielectric are

immersed in a carrier of another dielectric with a different index of refraction. The role of the

scaling factor % is similar to that of the surface albedo parameter in Lambert’s law. For very

precise results, one should compute the value of % using the quite complex procedure given

in [34]. However, the value of % is very nearly constant over most incidence and reflectance

angles [34]. Hence, in this paper we treat % as a constant.

Retroreflection: We can also develop a simplified version of the Wolff model when

~L ≈ ~V . When θr = θi = θ the two Fresnel terms are identical and the radiance simplifies to

ID(θ, n) = cos(θ)[1 − F (θ, n)]2 (6)

The Fresnel term has the effect of depressing the radiance for near-normal incidence. In Fig.

2.a we plot the radiance versus incidence angle using Eq. (6) for different values of n. The

greater the index of refraction, the greater the reduction in the near-normal reflectance.

The Fresnel term F (θ, n) given by Eqs. (4) and (5) is shown in Fig. 2.b by a solid curve.

There are a number of features of the curve that deserve comment, since they are pertinent to

the correction process. The first of these is that the Fresnel term is approximately constant

and given by Fc(n) = (1 − n)2/(1 + n)2 until a shoulder is reached. At the shoulder, it

increases rapidly to unity when θ = π/2. We can approximate the shoulder by a straight-

line whose slope is given by the gradient of the Fresnel term at θ = π/2. To first order,

the equation of this line is Fs(θ, n) = 1 + 2(θ − π/2)(n2 + 1)/√

n2 − 1. The position of the

shoulder can be approximated by the intersection of the two lines. For small values of n− 1,

the position of the shoulder is approximately given by θs = π/2− (√

n − 1/2√

2). The larger

the refractive index of the material n, the larger the value Fc(n). As a result, the effect of

the correction process is simply to scale the Lambertian reflectance by a constant amount

(1 − Fc)2 until the shoulder is reached. We show this behavior in Fig. 2.b by plotting the

term [1 − F (θ, n)]2 versus incidence angle as a dashed curve.

2.3 Combined Model of Wolff-Nayar-Oren

Wolff et al. [34] have commented on the fact that the two diffuse reflectance models, i.e.

the Oren-Nayar model for rough surfaces [23] and the Wolff model for smooth surfaces [32],

are complementary in their applicability to surfaces with different roughness properties.

10

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60

0.2

0.4

0.6

0.8

0.9

Incidence angle (radians)

Rad

ianc

e1.5

1.71.9

2.0(a)

0 10 20 30 40 50 60 70 80 900

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Fres

nel t

erm

( −

) and

the

Wol

ff te

rm (

− −

)

Incidence angle (degrees)

(b)

Figure 2: (a) Surface radiance versus θi using the Wolff model (Eq. 6) for n = 1.5, 1.7, 1.9, 2.0; (b)

Fresnel term (Eqs. 4-5; solid) and the term [1 − F (θ, n)]2 (dashed) versus θi for n = 1.4.

The Oren-Nayar model relies on the assumptions that the surface is composed of V-groove

microfacets and that the reflectance from each microfacet is Lambertian. The Wolff model,

on the other hand, assumes that there is subsurface optical scattering and air-dielectric

boundary conditions. These two effects are modelled by the Fresnel scattering term [34].

Hence, they [34] have suggested a methodology for how these two models can be combined, by

making the assumption that each V-groove microfacet reflects according to the Wolff model

(Eq. 3). Based on this assumption, a simple approximation to the diffuse reflectance model

that accounts for the observed empirical data over a broader physical range of roughness

conditions is obtained by replacing A in Eq. (1) with the quantity

C = A[1 − F (θi, n)]{1 − F (sin−1[sin θr/n], 1/n)} (7)

for an appropriate range of σ. This modification of the reflectance model is important

since radiance from surfaces with intermediate roughness exhibits a combination of effects

produced by both internal scattering and external roughness conditions.

Retroreflection: We can also simplify the combined diffuse reflectance model to the case

where ~L ≈ ~V . The simplified radiance is

ID(θ, σ, n) = A[1 − F (θ, n)]2 cos(θ) + B sin2(θ) (8)

Hence, the Fresnel term reduces the contribution from the Lambertian component at near-

normal incidence. The sin2 θ term is not affected, and results in limb brightening. In Fig.

3 we show the radiance versus the incidence angle using the combined model. The different

11

curves are for different roughness values in the range [0.01,0.3]. As the roughness increases,

then so the proportion of radiance at large incidence angles increases, and the departure

from the Lambertian case becomes more marked. For small values of σ the combined model

becomes identical to the Wolff model (Fig. 2.a). For large values of σ, on the other hand,

it does not behave exactly as the Oren-Nayar model. The combined model has an inflection

point close to θ = π/2, whereas the Oren-Nayar model does not exhibit this feature.

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Incidence angle (radians)

Rad

ianc

e

0.010.1

0.150.20.250.3

Figure 3: Surface radiance versus θi using the combined model (Eq. 8) for n = 1.7 and σ =

0.01, 0.1, 0.15, 0.2, 0.25, 0.3 rd.

3 Radiance Correction Process

Our overall aim is to use the diffuse reflectance models discussed in the previous sections

of this paper to perform radiance correction and hence recover the Lambertian component.

With corrected Lambertian images to hand, we can apply conventional SFS techniques to

recover surface shape information. There are of course many algorithms which can be used

for this purpose. However, here we use our recently developed SFS method outlined in [37].

This method is an extension of the Worthington and Hancock algorithm [6]. The surface

normals are constrained to fall on a cone whose axis points in the light-source direction

and whose apex angle is the inverse cosine of the corrected Lambertian component. In this

paper, we restrict our attention to the retroreflection case where the viewer and light-source

directions are co-incident. This means that the apex angle of the cone and the zenith angle

of the surface normal are identical. The zenith angle is the inverse cosine of the corrected

Lambertian radiance, i.e. θ = cos−1(IL). The position of the surface normal on the cone, i.e.

12

its azimuth angle φ, is free to vary. We initialize the azimuth angles of the surface normals

so that they point in the direction of the local image gradient. The surface normal directions

are iteratively updated by smoothing their directions and reprojecting the smoothed surface

normal directions onto the nearest position on the relevant cone. The smoothing process

adopted here involves aligning the directions of the local Darboux frames using the field

of principal curvature directions. Hence, we circumvent the problem of solving an under-

constrained inverse reflectance equation to recover surface normal direction. It is this inverse

problem that renders the non-Lambertian reflectance models intractable. By utilizing the

field of principal curvature directions we recover more accurate surface normals both at

parabolic locations on the surface and at the locations of surface discontinuities.

3.1 Closed Form Solution

The Oren-Nayar model can be inverted to recover the cosine of the incidence angle, and

hence the effective Lambertian radiance as follows. Suppose that at each image pixel (i, j)

the diffuse (non-specular) radiance ID consists of two components. The first of these is

a Lambertian component A cos θ. The second is the non-Lambertian component B sin2 θ

which takes on its maximum value where θ = π/2, i.e. close to the occluding boundary.

To perform radiance correction, we proceed as follows. At every pixel location, we use Eq.

(2) to estimate IL = cos θ using the measured radiance and solving the quadratic equation

ID = B + A cos θ − B cos2 θ for cos θ. The solution is

cos θ =A

2B

1 ±(

1 − 4B

A2(ID − B)

)

1

2

(9)

Since the Lambertian component is proportional to cos θ, this provides a means by which we

can perform direct radiance correction and remove the effects of rough surface reflectance.

The main problem with using Eq. (9) to estimate the Lambertian component is that

there are sometimes two possible solutions for each value of ID. This problem is illustrated

in Fig. 4.a which shows ID as a function of cos θ (for σ = 0.9rd); here the solutions are

found where the horizontal line corresponding to the radiance ID intersects the parabolic

curve. The value of cos θ must fall in the interval [0, 1], and hence we can rule-out solutions

that fall outside this interval. When cos θ = 0 then ID = B, and when cos θ = 1 then

ID = A. Moreover, the maximum point of the curve occurs at cos θ = A/2B which gives

ImaxD = B + A2/4B2. Fig. 4.b shows A and B as a function of σ. It is clear from the plot

13

0 10.35

Cosine of incidence angle

Sur

face

rad

ianc

e

(A−B)/B A/(2B)

B

A

B+(A/2B)(A/2B)

(a)

0 0.25 0.622 1 1.25 1.50

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Roughness parameter (radians)

A (s

olid

) and

B (d

ashe

d)

A

B

(b)

Figure 4: Inverse solution for the Oren-Nayar model using Eq. (9): (a) surface radiance versus

cos θ for σ = 0.9rd; (b) parameters A and B versus roughness parameter σ.

that A > B for all values of σ. Moreover, as σ becomes large then A tends to 0.5 and B

tends to 0.45. Provided that B < ID < A, then there is a single solution, and this lies in the

interval 0 < cos θ < (A−B)/B. When A < ID < ImaxD , then there are two possible solutions

placed symmetrically about cos θ = A/2B. When the quantity A/2B > 1 then only one of

the solutions gives a meaningful value of cos θ, and this is the one associated with the minus

sign in Eq. (9). The condition A/2B > 1 is satisfied when σ < 0.622rd; this is a rather large

value and corresponds to the extremes of roughness encountered in our experiments. Hence,

multiple solutions are only encountered when σ > 0.622rd for A < ID < ImaxD . This case

can be seen in Fig. 4.a where σ = 0.9rd. As the roughness parameter becomes large, then

A/2B approaches its limiting value of 5/9. Also, we note that when σ becomes small, i.e.

B becomes small then cos θ = (ID −B)/B. Finally, it is clear from Fig. 4.a that there is no

closed form solution for values of ID < B.

3.2 Lookup Table Solution

Although the problem of obtaining the corrected Lambertian radiance IL = cos θ in terms

of non-Lambertian diffuse radiance ID using the Oren-Nayar model is analytically tractable

(Section 3.1), the corresponding problems for the Wolff model and the combined model are

analytically intractable. For all three models, we therefore adopt a lookup table approach as

a practical alternative. To do this we tabulate cos θ as a function of the computed radiance

from each model. Since the Lambertian component is proportional to cos θ, this allows us to

14

perform Lambertian correction using the measured radiance values. In practice, the larger

the number of incidence angles tabulated (0 ≤ θ ≤ π/2), the more precise the correction.

For the Oren-Nayar model, using an approximate value of the surface roughness σ = σ0,

the lookup table is computed from Eq. (2). For the Wolff model, the radiance values are

almost identical for most dielectrics. This applies when the index of refraction n is in the

range (1.4 ≤ n ≤ 2.0). To give some idea of the physical significance of this restriction, glass

has a refractive index of about 1.5 while diamond has a refractive index which is even larger

than 2. Hence, we can use the center-range value (i.e. n = 1.7) when we compute the lookup

table using Eq. (6). Finally, for the combined model, the lookup table can be computed

using a value for the index of refraction and a value for the surface roughness using Eq. (8).

By assuming that ~L ≈ ~V , all three models can be made amenable to our lookup table

approach. The method is usable in practice since, like Lambert’s law, the brightness decreases

monotonically with increasing incidence angle (as shown in Figs. 1-3). As a result the

reflectance functions appearing in Eqs. 2, 6 and 8 are injective and invertible. Each measured

brightness value I0 is related to a single value of incidence angle θ0 and hence to a single

corrected Lambertian component cos(θ0).

4 Experiments

In this section we provide an experimental investigation of the correction process. We com-

mence with a study on synthetic data aimed at providing ground-truth, and then furnish

some results for real-world objects. In both cases the objects under study are illuminated in

the viewing direction.

4.1 Synthetic Data

In this section, we explore the use of the correction process on synthetic images with known

ground-truth surface normals. Here we experiment with cylindrical and spherical surfaces.

4.1.1 Cylindrical Surface

In Fig. 5.a we show the ground-truth elevation data for a cylindrical surface. Fig. 5.b shows

the corresponding Lambertian image. Using the ground-truth data, we generate three non-

Lambertian images using the Wolff (n = 2.0), the Oren-Nayar (σ = 0.52rd) and the combined

15

model (n = 1.7 and σ = 0.2rd). These images are shown in the top row of Fig. 6. We then

apply the Lambertian SFS method to these images and reconstruct the corresponding height

maps using the Frankot and Chellappa method [17], and the results are shown in the bottom

row of Fig. 6. Comparing the recovered height maps with the original surface in Fig. 5.a,

it is clear that there are significant shape difference, and these are attributable to errors in

the surface normal directions that result if SFS is applied to the uncorrected images. In Fig.

7 the top row shows the corrected Lambertian images when (from-left-to-right) the Wolff,

Oren-Nayar, and combined models are used. The lookup table parameters are identical to

those used in Fig. 6. In the second row of Fig. 7 we show the surfaces recovered from the

corrected images. The shape of the recovered surfaces is greatly improved.

Figure 5: (a) The ground-truth height map; (b) the ground-truth Lambertian image.

Figure 6: Top row: Images rendered using the Wolff (left), Oren-Nayar (middle) and combined

(right) models; bottom row: the corresponding height maps reconstructed using the surface normals

obtained by applying Lambertian SFS to the non-Lambertian images (in the top row).

In Fig. 8 we show plots of the brightness across transverse sections of the cylinders

shown in Figs. 6 and 7. Here the solid curves correspond to the initial uncorrected non-

Lambertian images, while the dashed curves correspond to the corrected Lambertian images.

16

Because of the large surface roughness value used in the Oren-Nayar model, the solid curve

(uncorrected) in Fig. 8.b exhibits the largest departure from the dashed curve (corrected).

In Fig. 8.a, it is important to note that despite relatively small differences between the

reflectance predictions by the Wolff model and Lambert’s law, the effect on the recovered

surface height is significant. This experiment illustrates the effectiveness of the correction

process and its usefulness for surface shape recovery.

Figure 7: Top row: corrected Lambertian images (by applying the table lookup method to the

images shown in Fig. 6) using the (a) Wolff, (b) Oren-Nayar and (c) combined models (from-

left-to-right); bottom row: the corresponding height maps reconstructed using the surface normals

obtained by applying Lambertian SFS to the corrected images (in the top row).

0 50 100 150 200 2500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

(a)

0 50 100 150 200 2500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

(b)

0 50 100 150 200 2500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

(c)

Figure 8: Transverse brightness plots corresponding to the non-Lambertian (solid) and the cor-

rected Lambertian (dashed) images using the (a) Wolff, (b) Oren-Nayar and (c) combined models.

4.1.2 Spherical Surface

The surface studied here consists of a large elliptical dome surrounded by four smaller ones.

In Fig. 9.a we show the elevation or height data for the synthetic surface, while Fig. 9.b

shows the field of ground-truth surface normals. In Fig. 10, we show the result of rendering

17

the synthetic surface using the four reflectance models studied in this paper. From left to

right, the panels show the images obtained using the Lambertian model, the Wolff model

(n = 1.7), the Oren-Nayar model (σ = 20◦), and the combined model (n = 1.7 and σ = 20◦).

In Fig. 11, we illustrate the effect of applying the correction process to the synthetic

images. In the top row the panels from left to right show the corrected Lambertian images

obtained by applying the table lookup method to the synthetic images generated using the

Wolff model, the Oren-Nayar model and the combined model. In the bottom row, the panels

show the difference between the corrected images and the original images (the darker the

image location, the greater the difference). In all three cases, the effects of the correction

process are greatest near the occluding limb of the object. The values of the parameters used

for constructing the lookup tables are different from those used when creating the images of

Fig. 10. Our aim in doing this is to illustrate that even using imprecise values of the surface

parameters, the method gives good results. The lookup table parameters are n = 2.0 in the

Wolff, σ = 10◦ in the Oren-Nayar, and, n = 1.7 and σ = 10◦ in the combined model.

To show that the corrected images are closer to the ground-truth Lambetian image than

the uncorrected ones, in Fig. 12 we compare the corresponding difference images. The top

row shows the differences between the ground-truth Lambetian image and the normalized

uncorrected images of Fig. 10, whereas the bottom row shows the differences with the nor-

malized corrected images of Fig. 11. From left-to-right the columns show the results obtained

with the Wolff, Oren-Nayar, and combined models. Using the Wolff model the corrected im-

ages show negligible differences, while using either the Oren-Nayar or the combined model,

the differences are smaller but non-negligible. These two models are reasonably sensitive

to the roughness parameter used. Hence with imprecise values of the roughness parameter,

although the corrected radiance is closer to the Lambertian radiance, small differences are

still expected. However, the Wolff model does not have a roughness parameter and is not

very sensitive to the index of refraction chosen.

In Fig. 13 we illustrate the effect of the correction process on the arc-cosine (ACS) er-

ror for the estimated surface normals. Each panel in the plot shows the image of ACS =

cos−1( ~NG. ~NE), where ~NG is the unit ground-truth surface normal and ~NE is the estimated

unit surface normal at the corresponding image location. To improve visualization, we show

negative images of the ACS error (the darker the point, the larger the error). The panels

in the top row show the ACS error when the SFS algorithm is applied to the uncorrected

18

0 20 40 60 80 100 1200

20

40

60

80

100

120

Figure 9: Synthetic surface: elevation data (left); field of surface normal directions (right).

Figure 10: Images of the surface of Fig. 9 rendered (from-left-to-right) using the models of

Lambert, Wolff (n = 1.7), Oren-Nayar (σ = 20◦), and Wolff-Nayar-Oren (n = 1.7 and σ = 20◦).

Figure 11: Top row: corrected images corresponding to the non-Lambertian images of Fig. 10

using the Wolff, Oren-Nayar, and combined models (from-left-to-right); bottom row: differences

between uncorrected and corrected images.

images generated with the Wolff, Oren-Nayar and combined models (left-to-right). In the

second row, we show the ACS error plots obtained when the SFS algorithm is applied to the

corrected images. It is clear from the plots that the greatest error occurs at the cusp between

the surfaces, where the SFS algorithm results in some oversmoothing of the field of surface

normals. It is important to note that the problem of shape recovery at intersections between

surfaces with singular points and discontinuities cannot be overcome just by correcting re-

flectance properties. For points of the surface which do not exhibit such discontinuities,

the errors corresponding to the corrected images are smaller than those corresponding to

19

Figure 12: Differences between the ground-truth Lambetian image and the uncorrected images of

Fig. 10 (top row), and the corrected images of Fig. 11 (bottom row), with the Wolff, Oren-Nayar,

and combined models (from-left-to-right).

the uncorrected images. Also, while the errors are greatest at the object boundary before

correction, the distribution in error across the surface is relatively uniform after correction.

Figure 13: ACS error images for the ground-truth and the recovered surface normals corresponding

to the uncorrected images of Fig. 10 (top row) and the corrected images of Fig. 11 (bottom row).

To analyze the behavior of the method in more detail, we compute the total-mean-square

(TMS) error between the field of ground-truth unit surface normals ~NG and the field of

estimated unit surface normals ~NE. We compute the TMS error using the formula TMS =

Σi,j{( ~NGij (x) − ~NE

ij (x))2 + ( ~NGij (y) − ~NE

ij (y))2 + ( ~NGij (z) − ~NE

ij (z))2} where x, y and z denote

the vector components and i, j denote the image pixel location. In Fig. 14 we show the TMS

error between the estimated and ground-truth surface normals as a function of iteration

number. In each plot, the dashed curve corresponds to the uncorrected images, while the

20

solid curve corresponds to the corrected images. In each case the error is significantly lower

after the correction. For all cases we use 30 iterations in the SFS algorithm. To give some

idea of the scale of the TMS error, here we compute the maximum error. This maximum error

occurs when every estimated surface normal is in the opposite direction to the corresponding

ground-truth direction. For the surface used in this experiment (Fig. 9) TMSmax = 50292.

0 5 10 15 20 25 30550

600

650

700

750

800

850

Iteration

Nor

mal

Err

or

0 5 10 15 20 25 30400

500

600

700

800

900

1000

1100

1200

Iteration

Nor

mal

Err

or

0 5 10 15 20 25 30400

500

600

700

800

900

1000

1100

Iteration

Nor

mal

Err

or

Figure 14: TMS surface normal error versus iteration number when applying Lambertian SFS

method [37] to the uncorrected (dashed) and corrected (solid) images shown in Figs. 10-11 using

the Wolff, Oren-Nayar and combined models (from-left-to-right).

4.2 Real-World Data

Our experiments with real-world data are divided into four parts. We commence by illus-

trating the effect of the correction process. Second, we analyze the effects of the correction

process on the surface normal directions. Third, we show results on surface height recovery.

Finally, we present a study on cylindrical surfaces which allow the results of the correction

process to be compared with ground-truth.

The real-world images used in our experiments have been captured using an Olympus 10E

camera. Each surface has been imaged under controlled lighting conditions in a darkroom.

The objects have been illuminated using a single collimated tungsten light-source. To exper-

iment under the conditions where ~L ≈ ~V , the light-source and camera are slightly displaced

in the vertical plane. For dielectric shiny surfaces (e.g. porcelain objects) both specular and

diffuse components are present. The specular component must therefore be removed before

we can perform our radiance correction on the diffuse component. One reason for this is

that local specularities may be misidentified as high curvature surface features. To subtract

the specular component we use the method that we have reported earlier in [38]. However,

for matte dielectric surfaces (e.g. terra-cotta objects) the diffuse component is the dominant

21

one. Hence, specularity subtraction is not necessary. The images (raw images in the case of

matte objects and those to which specularity subtraction has been applied for shiny objects)

used as input to the correction process is referred to as “diffuse” images.

4.2.1 Results of Radiance Correction

In this section we show the results of performing surface radiance correction by applying

table look-up with the Oren-Nayar, Wolff and combined models. Since, the differences in

brightness resulting from the correction process are small, for the purposes of visualization

we magnify the differences by a factor K (typically K = 10). In the difference images, the

darker the pixel, the larger the difference value.

In Figs. 15 we show some results for three rough terra-cotta objects. In the top row of

Fig. 15, we show the original (diffuse) images. In the second row, we show the recovered

Lambertian images obtained by lookup table using the Oren-Nayar model. The third row

shows the images of the differences between the original images and the recovered images.

Finally, in the fourth row we show transverse brightness sections across approximately cylin-

drical regions of the objects. Here the solid curve is the original brightness while the broken

curve is the corrected brightness. Note that the broken curve is lower than the solid curve.

Although the correction process has an effect at almost every location on the surface, the

differences are most marked where the inclination of the object surface is steepest.

In Figs. 16, we turn our attention to shiny porcelain objects. Here we must first perform

specularity subtraction. Hence, in the second row of Fig. 16 we show the result of performing

specularity removal using the method described in [38]. These are the diffuse images used as

inputs to the correction process. The third row shows the specular component reconstructed

using the Torrance-Sparrow model [25, 38]. In the fourth row we show the recovered Lamber-

tian images obtained using the Wolff model. The fifth row shows the images of the differences

between the recovered Lambertian component and the diffuse component. When comparing

these results to those obtained using the Oren-Nayar model (Fig. 15), it is clear that the

radiance correction is greatest for the Wolff model. Also, the differences obtained using the

Wolff model are sharper than those obtained using the Oren-Nayar model. Finally, the sixth

row shows transverse brightness sections across cylindrical surface regions. Here the broken

curve (corrected) is higher than the solid curve (diffuse).

In Figs. 17, we show the results for two objects with smooth and rough surfaces when

22

Figure 15: Radiance correction using the Oren-Nayar model for terra-cotta objects: raw images

(top); corrected images (second); difference images (third); transverse brightness sections (bottom).

the combined model is used for radiance correction. The first row in Fig. 17 shows the

original images (diffuse component for the porcelain urn) while the second row in each

figure shows the recovered Lambertian components. The third row shows the differences

between the recovered Lambertian images and the diffuse images. Finally, the bottom row

shows transverse brightness sections. Here, for the locations close to the occluding object

boundary, the broken curve (corrected) is higher than the solid curve (diffuse). For other

surface locations the pattern is reversed. The main features to note from the difference

images are as follows. First, the correction on the limbs of the objects is most marked for

the porcelain urn which has a smooth surface. However, the amount of limb correction for the

terra-cotta bear is significantly smaller. Second, the amount of rough reflectance correction

is greatest for the terra-cotta bear, and, is least marked for the porcelain urn. Note that the

surface of the urn is not perfectly smooth and that there is a small roughness component.

23

Figure 16: Recovering Lambertian images (fourth) by removing specularities (third) from the raw

images (top) [38], and, applying the correction process using the Wolff model to the separated diffuse

images (second) of porcelain objects; differences (fifth); transverse brightness sections (bottom).

4.2.2 Surface Normal Directions

In this section we turn our attention to the surface normal directions recovered using our

Lambertian SFS algorithm [37]. In Fig. 18 we show the fields of surface normals, or needle-

maps, extracted from the images shown in Figs. 15-17. In the left-hand column of Fig. 18,

we show the needle-maps obtained when the Oren-Nayar model is applied (Fig. 15, terra-

cotta teapot), the middle column those obtained when the Wolff model is used (Fig. 16,

24

Figure 17: Radiance correction using the combined model for a smooth and a rough object: diffuse

images (top); corrected images (second); differences (third); brightness sections (bottom).

porcelain bear), and the right-hand column those obtained when the combined model is used

(Fig. 17, porcelain urn). The results in the top row correspond to the “diffuse” images, and

those in the middle row correspond to the corrected Lambertian images. We also investigate

the differences in the fields of surface normals extracted from the “diffuse” images and the

corrected images. In the bottom row we show the field of surface normal differences. The

most significant differences occur at the steepest locations on the surface. In addition, it

should be noted that the needle-maps reflect the fine detail of the surfaces.

We now focus in more detail on the effects of the correction process on the surface normal

directions. In qualitative terms, when the image becomes brighter then the apex (opening)

angle of the Lambertian cone becomes smaller. The reason for this is that the Lambertian

SFS method constrains the surface normals to fall on a cone whose apex angle is the arc-

cosine of the brightness. The axis of the cone points in the light-source direction, and since

~L = ~V , the apex angle of the cone is equal to the zenith angle θ. Hence the zenith angle θL =

cos−1(IL) of the surface normal decreases. When the image becomes darker, on the other

25

0 50 100 150 2000

50

100

150

200

0 20 40 60 80 100 120 140 160 180 2000

20

40

60

80

100

120

140

160

180

200

0 20 40 60 80 100 120 140 160 180 2000

20

40

60

80

100

120

140

160

180

200

0 50 100 150 2000

50

100

150

200

0 20 40 60 80 100 120 140 160 180 2000

20

40

60

80

100

120

140

160

180

200

0 20 40 60 80 100 120 140 160 180 2000

20

40

60

80

100

120

140

160

180

200

0 50 100 150 2000

50

100

150

200

0 20 40 60 80 100 120 140 160 180 2000

20

40

60

80

100

120

140

160

180

200

0 20 40 60 80 100 120 140 160 180 2000

20

40

60

80

100

120

140

160

180

200

Figure 18: Needle maps obtained by applying the Lambertian SFS method [37] to the uncorrected

(top) and the corrected (center) images shown in Figs. 15, 16 and 17; difference (bottom).

hand, then the zenith angle increases. Fig. 19 explores the effect of the correction process

on the azimuth and zenith angles for the terra-cotta bear using the combined model. The

left-hand plot shows the zenith angle θL = cos−1(IL) extracted from the corrected radiance,

as a function of the zenith angle estimated from the uncorrected diffuse image radiance,

i.e. θr = cos−1(ID). The center panel shows a plot of the azimuth angle estimated from the

corrected radiance, as a function of the azimuth angle estimated from the uncorrected diffuse

radiance. Since the normal is constrained to fall on a cone (as described above), in the zenith

angle plot there is no scatter and the data points trace out a curve. The Lambertian zenith

angle is greater than the diffuse zenith angle for small and intermediate angles. For large

angles, however, the pattern is reversed. The azimuth angle plot, on the other hand, does

exhibit scatter, but there is a clear regression line. The reason for this is that the surface

normals are free to rotate about the cones subject to smoothness constraints. To investigate

the stability of the surface normals under changes in azimuth angle, in the right-hand panel

26

0 10 20 30 40 50 60 70 80 900

10

20

30

40

50

60

70

80

90

−200 −150 −100 −50 0 50 100 150 200−200

−150

−100

−50

0

50

100

150

200

0 10 20 30 40 50 60 70 80 90−200

−150

−100

−50

0

50

100

150

200

Figure 19: Scatter plots for the incidence zenith (left) and azimuth (center) angles (in degrees)

and azimuth difference angles (right) extracted from the surface normals for the terra-cotta bear.

of Fig. 19 we show the difference in azimuth angle extracted from the “diffuse” and corrected

images as function of the zenith angle extracted from the corrected image. It is clear that the

large azimuth angle differences occur at small zenith angles. Hence, these are associated with

small changes in surface normal direction since the opening angle of the cone is small. The

largest differences in azimuth angle occur at intermediate zenith angles. This means that

the combined model results in significant differences in surface normal direction. Although

this effect is mainly attributable to the larger differences in corrected radiance, it may also

reflect problems associated with the curvature dependant smoothing employed in the SFS

algorithm. This uses the principal curvature direction to adjust the azimuth angle of the

surface normal. When the surface is umbilic (e.g. at spherical locations) or hyperbolic (e.g.

at saddles) there are singularities in the field of principal curvature directions and these in

turn may lead to unstable azimuth angle estimates. Note also that SFS results in small

differences in azimuth angle near object limbs. The reason for this is that the boundary

condition constrains surface normal to be perpendicular to the occluding boundary.

4.2.3 Height Map Reconstruction

In Fig. 20 we show the results of applying the Frankot and Chellappa algorithm [17] to the

surface normals extracted from three real-world images (shown in Fig. 18). The different

columns show the results for the porcelain bear, the porcelain urn and the terra-cotta teapot.

The top row of the figure shows the surface height maps recovered from the input “diffuse”

images (the top row of Fig. 18). The second row shows the results obtained from the

corrected images (the middle row of Fig. 18); we have used the Wolff model for the bear, the

27

combined model for the urn and the Oren-Nayar model for the teapot. The third row shows

the height difference between the surfaces recovered from the input “diffuse” and corrected

images. For comparison, the bottom row shows height data for the objects obtained using

a Polhemus Fastscan Cobra range sensor. The surfaces recovered are detailed and appear

to reflect the shapes of the underlying object very well. Much of the surface structure is

recovered, and sometimes this is better than that delivered by the range sensor. The reason

for this is that the range sensor relies on the stereoscopic reconstruction of depth from a laser

stripe. This stripe is easily detected on matte surfaces, but the reconstruction does not work

well on shiny surfaces since it is confused by reflections. The height differences between the

surfaces extracted from the input “diffuse” and corrected images are greatest at the object

boundaries.

4.2.4 Comparison with Ground Truth

In this section we compare the results of the correction process with ground-truth data. To

do this we use a cylindrical object, which allow the evaluation of results. We have aligned the

cylinder so its axis of symmetry is vertical (i.e. aligned with the y-axis). If d is the measured

diameter of the cylinder on the image, x0 is the horizontal coordinate of the symmetry axis

and x is the measured horizontal coordinate of the point on the cylinder, then the predicted

unit surface normal at the measured point is ~Ncyl(x) = (cos γ, 0, sin γ)T where cos γ =

2(x − xo)/d. The predicted Lambertian component on the cylinder is Icyl(x) = 2(x − xo)/d.

In the top row of Fig. 21, the left-hand panel shows the raw image of a strawboard cylinder.

We show the corrected images obtained using the Oren-Nayar model with σ = 0.1rd in the

center panel and with σ = 0.12rd in the right-hand panel. In the bottom row of the figure,

we show the difference (K = 5) between the raw and corrected images in the top row. Here,

the darker a point on a difference image, the higher the difference value. The effect of using

the larger roughness value is to increase the difference, especially near the boundary of the

cylinder. Fig. 22 illustrates the difference (K = 1) between the corrected Lambertian image

and the predictions delivered by the fit to the cylinder geometry. In the left-hand panel of

Fig. 22 we show the Lambertian component predicted by the cylinder fit. The remaining

panels in Fig. 22, from left to right, show the difference between the cylinder prediction and

the images in the top row of Fig. 21, i.e. the raw image, and the Lambertian corrections

obtained with σ = 0.1rd and σ = 0.12rd. The greatest difference occurs for the raw image.

28

Figure 20: Height maps reconstructed using the Frankot-Chellappa method [17] for surface nor-

mals corresponding to input “diffuse” images (top row) and corrected Lambertian images (second

row) using Wolff (left column), combined (middle column) and Oren-Nayar (right column) models;

difference between height maps (third row); elevation data using a range finder (bottom row).

In Fig. 23.a, we compare the recovered Lambertian radiance using the Oren-Nayar model

with the prediction of Lambert’s law for a cross section on the strawboard half-cylinder.

The plots show the normalized radiance as a function of the incidence angle γ computed

29

Figure 21: Top row: (a) raw image of a strawboard cylinder, corrected images using the Oren-

Nayar model for (b) σ = 0.1 and (c) σ = 0.12rd (left-to-right); bottom row: magnified differences

between the raw and corrected images.

Figure 22: From-left-to-right: ground-truth Lambertian image for a fitted cylinder, and, its abso-

lute differences with the raw image and the two corrected images of Fig. 21, respectively.

from the fit of the cylinder. In the plot, the solid curve is the measured radiance, the dash-

dot curve is the prediction of Lambert’s law, the dashed curve is the recovered Lambertian

radiance with σ = 0.1rd and the dotted curve that with σ = 0.12rd. The closer the corrected

Lambertian radiance to the Lambertian curve, the more successful the correction process.

Here, it is the dotted curve which is the closest to the Lambertian curve. Hence, the Oren-

Nayar model with σ = 0.12 rd results in a better correction than that with σ = 0.1rd. It

is worth mentioning that for higher values of the roughness parameter, i.e. σ > 0.12rd, the

difference between the corrected radiance and the ground-truth increases.

In Fig. 23.b, we show plots of the total error between the recovered surface normals and

the surface normals predicted by the cylinder fit, as a function of the iteration number for

the SFS algorithm. The solid curve is for the raw image, the dashed curve for the corrected

image obtained using the Oren-Nayar model with σ = 0.1rd, and the dash-dot curve that

obtained with σ = 0.12rd. It is the dash-dot curve that corresponds to the lowest error.

30

0 10 20 30 40 50 60 70 80 900

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Incidence angle (degrees)

Rad

ianc

e

Lambertian

Corrected (O−N, 0.1)

Corrected (O−N, 0.12)

Measured radiance

60 90 120 150160

180

200

220

Iteration

Nor

mal

Err

or

Raw image

Corrected (O−N, 0.1)

Corrected (O−N, 0.12)

Figure 23: Left: normalized raw radiance for the strawboard cylinder versus θi compared to the

corrected radiance using the Oren-Nayar model (σ = 0.1 and σ = 0.12rd) and Lambert’s law; right:

TMS error versus iteration number obtained by applying SFS to the images of Fig. 21.

5 Conclusions

In this paper, we have investigated how to perform radiance correction so that a Lambertian

shape-from-shading (SFS) method can be applied to both shiny and rough objects. We have

analyzed three diffuse reflectance models. The Wolff model is for shiny surfaces, the Oren-

Nayar model is for rough surfaces and the Wolff-Nayar-Oren model combines rough and

smooth scattering effects. There are a number of obstacles to the correction process. Firstly,

the models depend on both the direction of the incident light and the viewer. Second, even

when these two directions are identical, then the Wolff model and the Wolff-Nayar-Oren

model are not tractable because of the multiplicative nature of the Fresnel correction. To

overcome these problems, we construct a lookup table which allows the cosine of the incidence

angle, and hence the Lambertian component, to be recovered from the measured brightness.

It is interesting to note that in the case of the Oren-Nayar model, which involves an additive

rather than multiplicative correction of Lambert’s law, then the radiance correction can be

performed by inverting the radiance equation. To apply the reflectance models and perform

Lambertian correction, we need to know the surface roughness parameter and the index of

refraction appearing in the models. Here, we use approximate values of these parameters for

similar surfaces obtained from tabulations in the literature [23, 34].

We have derived simplified formulas for each model for the case of retroreflection to perform

radiance correction using table lookup. These simplified models are applicable when the angle

31

between the light-source and viewing directions is small. This condition is easily achievable

in practice. Under these conditions the simplified models are only dependent on the angle of

incidence. Finally, we experiment with a variety of real-world images of surfaces with different

scales of roughness. Here we show the results for our radiance correction process using the

three reflectance models. These results demonstrate that for surfaces with intermediate

roughness the results delivered by the separate rough or smooth reflectance models are poor.

However, the combined model of Wolff-Nayar-Oren provides reliable reflectance predictions

for a wide range of surfaces which can be characterized as having intermediate roughness.

We have also investigated the effect of the correction process on the surface normal directions

recovered using SFS. Our experiments show that different models can be used to correct the

radiance provided that the appropriate roughness conditions apply, and that the correction

process leads to improved surface normal estimates. Hence, we argue for the use of such

phenomenological reflectance models in applications including photometric stereo and surface

inspection.

Finally, it is important of note that the reflectance models studied in this paper are phe-

nomenological. Hence, the parameters are not directly related to the physical properties of

the surface. There is a considerable body of literature concerned with the detailed physical

modelling of rough surfaces. This work builds on the Kirchhoff theory [29] which has been

developed by Beckmann [29] and by Vernold and Harvey [30]. In the computer vision litera-

ture there have been some attempts to exploit this body of work for reflectance and texture

modelling [20]. This more thorough physics-based approach provides a means by which the

reflectance from surfaces of diverse roughness may be modelled. In a recent paper we have

used the Kirchhoff theory to develop reflectance models which can be used for rough surface

analysis tasks in computer vision. The main thrust of this work has been to use the the-

ory to develop ways of estimating surface roughness parameters [39]. The resulting models

are considerably more complicated than the phenomenological ones used here. However, we

have compared them to BRDF measurements, and they agree well with the data. In fact for

surfaces with intermediate roughness they outperform the phenomenological models studied

here. It is also worth noting that the wave scattering theory can also be used to perform

radiance correction, and [39] also explores this as a relatively minor component part of its

experimental evaluation.

32

References

[1] B.K.P. Horn, Shape from Shading: A Method for Obtaining the Shape of a Smooth Opaque

Object from One View, PhD Thesis, Department of Electrical Engineering, MIT, 1970.

[2] K.J. Krakauer, Computer Analysis of Visual Properties of Curved Objects, PhD Thesis, MIT

AI-TR, 19710.

[3] B.K.P. Horn and M.J. Brooks, Shape from Shading, MIT Press, Cambridge Massachusetts,

1989.

[4] R. Kozera and R. Klette, ”Finite Difference Based Algorithms for Linear Shape from Shading,”

Machine Graphics and Vision, 1997, pp. 157201.

[5] R. Zhang and M. Shah, “Iterative Shape Recovery from Multiple Images,” Image and Vision

Computing, vol. 15, 1997, pp. 801-814.

[6] P.L. Worthington and E.R. Hancock, “New Constraints on Data-closeness and Needle-map

consistency for SFS,” IEEE Trans. Pat. Anal. Mach. Int., vol. 21, no. 11, 1999, pp. 1250-1267.

[7] R. Kimmel and A.M. Bruckstein, “Tracking Level-sets by Level-sets: A Method for Solving

the Shape from Shading Problem”, Computer Vision and Image Understanding, vol. 62, no. 1,

1995, pp. 47-58.

[8] J. Oliensis and P. Dupuis, “A Global Algorithm for Shape from Shading”, Proc. IEEE Int’l

Conf. Comp. Vis., 1993, pp. 692-701.

[9] D.A. Forsyth and A. Zisserman, “Reflections on Shading”, IEEE Trans. Pat. Anal. Mach. Int.,

vol. 13, no. 7, 1991, pp. 671-679.

[10] M.J. Brooks, W. Chojnacki and R. Kozera, “Impossible and Ambiguous Shading Patterns”,

Int’l. J. Comp. Vis. vol. 7, no. 2, 1992, pp. 119-126.

[11] R. Zhang, P. Tsai, J.E. Cryer and M. Shah, “Shape from Shading: A Survey,” IEEE Trans.

Patt. Anal. Mach. Intel., vol. 21, no. 8, 1999, pp. 690-706.

[12] T. Rindfleisch, “Photometric Method for Lunar Topography,” Photogrammetric Engineering,

vol. 32, no. 2, 1966, pp. 262-277.

[13] B.W. Hapke, “A Theoretical Photometric Function for the Lunar Surface,” J. Geophysical

Research, vol. 68, no. 15, 1963, pp. 4571-4586.

33

[14] M. Minnaert, Photometry of the Moon, Chapter 6 in Volume 3 of Planets and Sattelites: the

Solar System, University of Chicago Press, IL, 1961, pp. 213-248.

[15] V. Fesenkov, Photometry of the Moon, in Physics and Astronomy of the Moon, New York,

Academic Press, 1962, pp. 99-130.

[16] B.K.P. Horn, “Hill Shading and the Reflectance Map,” Proceedings of the IEEE, vol. 69, no.

1, 1981, pp. 14-47.

[17] R.T. Frankot and R. Chellappa, “A Method for Enforcing Integrability in Shape from Shading

Algorithms,” IEEE Trans. Patt. Anal. Mach. Intel., vol. 10, no. 4, 1988, pp. 439-451.

[18] A.G. Bors, E.R. Hancock and R.C. Wilson, “Terrain Analysis Using Radar Shape-from-

Shading,” IEEE Trans. Patt. Anal. Mach. Intel., vol. 25, no. 8, 2003, pp. 974-992.

[19] K. Dana, S.K. Nayar, B.V. Ginneken and J.J. Koenderink, “Reflectance and Texture of Real-

World Surfaces,” Proc. IEEE Conf. Comp. Vis. Patt. Recog., 1997, pp. 151-157.

[20] B.V. Ginneken, M. Stavridi and J.J. Koenderink, “Diffuse and Specular Reflectance from

Rough Surfaces,” Applied Optics, vol. 37, no. 1, 1998, pp. 130-139.

[21] S. Magda, D.J. Kriegman, T.E. Zickler and P.N. Belhumeur, “Beyond Lambert: Reconstruct-

ing Surfaces with Arbitrary BRDFs,” Proc. IEEE Int’l. Conf. Comp. Vis., 2001, pp. 391-399.

[22] B.T. Phong, “Illumination for Computer Generated Pictures,” Communications ACM, vol.

18, 1975, pp. 311-317.

[23] M. Oren and S.K. Nayar, “Generalization of the Lambertian Model and Implications for

Machine Vision,” Int’l. J. Comp. Vis., vol. 14, no. 3, 1995, pp. 227-251.

[24] P. Kubelka, “New Contribution to the Optics of Intensely Light-Scattering Materials: Part I”,

J. Opt. Soc. Am., vol. 38, no. 5, 1948, pp. 448-457.

[25] K. Torrance and E. Sparrow, “Theory for Off-Specular Reflection from Roughened Surfaces,”

J. Opt. Soc. Am., vol. 57, 1967, pp. 1105-1114.

[26] J.M. Bennett and L. Mattsson, Introduction to Surface Roughness and Scattering, Optical

Society of America, Washington, D.C., 1999.

[27] J. Blin, “Models of Light Reflection for Computer Synthesized Pictures,” Computer Graphics,

vol. 11, no. 2, 1977, pp. 192-198.

34

[28] R.L. Cook and K.E. Torrance, “A Reflectance Model for Computer Graphics,” Computer

Graphics, vol. 15, no. 3, 1981, pp. 307-316.

[29] P. Beckmann and A. Spizzichino, The Scattering of Electromagnetic Waves from Rough Sur-

faces, Pergamon, New York, 1963.

[30] C.L. Vernold and J.E. Harvey, “A Modified Beckmann-Kirchoff Scattering Theory for Non-

paraxial Angles,” Scattering and Surface Roughness, Proc. SPIE, vol. 3426, 1998, pp. 51-56.

[31] H. Ragheb and E.R. Hancock, “Estimating Surface Characteristics using Physical Reflectance

Models,” Proc. IEEE Conf. Comp. Vis. Patt. Recog., vol. 2, 2003, pp. 177-184.

[32] L.B. Wolff “Diffuse Reflectance Model for Smooth Dielectric Surfaces,” J. Opt. Soc. Am. A,

vol. 11, no. 11, 1994, pp. 2956-2968.

[33] L.B. Wolff, “On The Relative Brightness of Specular and Diffuse Reflection,” Proc. IEEE

Conf. Comp. Vis. Patt. Recog., 1994, pp. 369-376.

[34] L.B. Wolff, S.K. Nayar and M. Oren, “Improved Diffuse Reflection Models for Computer

Vision,” Int’l. J. Comp. Vis., vol. 30, no. 1, 1998, pp. 55-71.

[35] N.S. Orlova, “Photometric Releif of the Lunar Surface,” Astron. Z, 1956, vol. 33, no. 1, pp.

93-100.

[36] B.W. Hapke, R.M. Nelson and W.D. Smythe, “The Opposition Effect of the Moon: The

Contribution of Coherent Backscatter,” Science, vol. 260, no. 23, 1993, pp. 509-511.

[37] H. Ragheb and E.R. Hancock, “Darboux Smoothing for Shape-from-Shading,” Pattern Recog-

nition Letters, vol. 24, nos. 1-3, 2003, pp. 579-595.

[38] H. Ragheb and E.R. Hancock, “A Probabilistic Framework for Specular Shape-from-Shading,”

Pattern Recognition, vol 36, no. 2, 2003, pp. 407-427.

[39] H. Ragheb and E.R. Hancock, “The Modified Beckmann-Kirchhoff Scatter Theory for Rough

Surface Analysis,” submitted to IEEE Trans. Pat. Anal. Mach. Int.

35