Efficient Estimation of Spatially Varying Subsurface Scattering Parameters

Sarah Tariq1,3,Andrew Gardner2, Ignacio Llamas1,3, Andrew Jones2, Paul Debevec2 and Greg Turk1

Georgia Institute of Technology1, Institute for Creative Technologies University of Southern California2,NVIDIA Corporation3

Abstract

We present an image-based technique to efficientlyacquire spatially varying subsurface reflectanceproperties of a human face. The estimated prop-erties can be used directly to render faces with spa-tially varying scattering, or can be used to estimatea robust average across the face. We demonstrateour technique with renderings of peoples’ faces un-der novel, spatially-varying illumination and pro-vide comparisons with current techniques. Our cap-tured data consists of images of the face from a sin-gle viewpoint under two small sets of projected im-ages. The first set, a sequence of phase-shifted pe-riodic stripe patterns, provides a per-pixel profile ofhow light scatters from adjacent locations. The sec-ond set of structured light patterns is used to obtainface geometry. We subtract the minimum of eachprofile to remove the contribution of interreflectedlight from the rest of the face, and then match theobserved reflectance profiles to scattering propertiespredicted by a scattering model using a lookup ta-ble. From these properties we can generate imagesof the subsurface reflectance of the face under anyincident illumination, including local lighting. Therendered images exhibit realistic subsurface trans-port, including light bleeding across shadow edges.Our method works more than an order of magnitudefaster than current techniques for capturing subsur-face scattering information, and makes it possiblefor the first time to capture these properties over anentire face.

1 Introduction

Rendering human faces realistically has been alongstanding problem in computer graphics and asubject of recent increased interest. There are sev-eral important areas that require the realistic render-ing of human faces, including computer games, an-imated feature films, and special effects for movies.One of the most crucial factors in creating convinc-

ing images of faces is realistic skin rendering. Thisis a hard problem because skin reflectance consistsof many complex components, including light inter-reflection and subsurface scattering.

Subsurface scattering is the phenomenon of lightentering one point on a surface of a material andscattering inside it before exiting at another point.In this process, light is not only scattered, but mayalso be partially absorbed by the material and theseeffects typically vary for different wavelengths. Vi-sually, subsurface scattering results in a softeningof the appearance of the material, color bleedingwithin the material, and diffusion of light acrossshadow boundaries. The human visual system caneasily notice the absence of these cues. Light dif-fusion across illumination boundaries is especiallyimportant for cinematic lighting, where faces areoften in partial lighting and the resulting shadowedges are soft and blurred.

The dipole diffusion model [6] has been pro-posed as a fast closed-form approximation for cal-culating the outgoing radiance due to subsurfacescattering, and is capable of producing very real-istic images. Rendering with this model, however,requires an estimate of the scattering and absorp-tion parameters of the skin. Current methods forcapturing these parameters require setups not suit-able for faces [6], or specialized devices [17]. Moreimportantly, these methods are too slow to capturethe parameters over an entire face, providing onlya sparse set of parameters. Skin, however, is not ahomogeneous material, and its scattering propertieschange across different people and across a face;cheeks have more blood, giving them a reddish ap-pearance, areas with whiskers tend to be darker, andthere may be uneven patches in the skin. Spatiallyvarying parameters are hence important for captur-ing the appearance of skin, and, as shown in [5],are already used in the entertainment industry forrendering realistic faces. Due to the absence of amethod for estimating spatially varying scatteringproperties of skin, these techniques could not use

independent spatially varying parameters (see Sec-tion 2 for details). Figures 4 and 7 show the impor-tance of using the spatially varying scattering pa-rameters estimated by our method in producing anaccurate image.

Our Approach. We propose a technique thatmeasures the spatially varying scattering propertiesof a subject, using just a camera and projector, withacquisition times under a minute. This time frameis small enough to permit a human subject to re-main sufficiently still. We reduce the number ofimages needed by making key assumptions aboutlight scattering in skin, which are that the scatteringis isotropic, local, and has low spatial frequency.

We estimate a point’s scattering properties usingaScattering Profilethat encodes the amount of lightthat scatters to it from nearby locations. We mea-sure one such profile per pixel (see Figure 2), us-ing information derived from an image of the sub-ject taken under shifting patterns of black and whitestripes. The spatially varying scattering propertiesof the face are derived from theseScattering Pro-files by matching them with a lookup table (LUT)of pre-rendered profiles with known absorption andscattering properties. The per-pixel estimated prop-erties and captured geometry are then used to renderthe face under new local illumination.

2 Background and Related WorkScattering Models Subsurface transport is de-scribed by the Bidirectional Surface ScatteringReflectance Distribution Function (BSSRDF), S,which relates the outgoing radianceLo(xo,−→wo) atsurface pointxo in the direction−→wo to the incidentflux φ(xi ,−→wi ) at surface pointxi from the direction−→wi :

dLo(xo,−→wo) = S(xi ,−→wi ;xo,−→wo)dφ(xi ,−→wi ) (1)

Given S, the outgoing radianceLo can be com-puted by integrating the incident radiance over in-coming direction and area:

Lo(xo,−→wo) =∫

2π

∫A

S(xi ,−→wi ;xo,−→wo)

Li(xi ,−→wi )(n·wi)dwidA(xi) (2)

The scattering of light in a material is dictatedby two optical parameters per wavelength. Theseare the scatteringσs and absorptionσa coefficients,which indicate the fraction of light that is scatteredor absorbed by the medium for each unit length ofdistance traveled by the light. The 8-dimensional

σ ′t reduced extinction coefficientσ ′

s+σaα ′ reduced albedoσ ′

s/σ ′t

σtr effective transport extinction coefficient√

3σaσ ′t

lu mean free path1/σ ′t

ld diffuse mean free path1/σtrzr distance from real source to surfaceluzv distance from virtual source to surfacelu(1+4A/3)r distance‖xo−xi‖dr distance fromxo to real source

√r2 +z2

r

dv distance fromxo to virtual source√

r2 +z2v

Table 1:Definition of variables.

BSSRDF can be evaluated accurately by solving theradiative transport equation, also known as the vol-ume rendering equation. This is an integral equa-tion and direct methods to solve it, including MonteCarlo simulation [4], finite element methods andphoton mapping, are relatively slow.

The diffusion approximation to light propagationin participating media states that for optically densematerial the behavior of the material is dominatedby multiple scattering [13] and directional depen-dence is negligible. Hence the BSSRDF can be ap-proximated faithfully by the 4D diffusion approx-imation, Rd(xo,xi), which depends only on the in-coming and outgoing points. [13] presents numer-ical solutions to solve the diffusion equation, butdoes not present a closed-form solution.

The dipole approximation to the diffusion modelis a fast closed-form solution introduced by [6]. ItevaluatesRd(‖xo−xi‖) for a half infinite slab of ho-mogeneous material as:

Rd(r) = α ′

4π

{zr

(σtr + 1

dr

)e−σtr dr

d2r

+

zv

(σtr + 1

dv

)e−σtr dv

d2v

}(3)

Given the scattering and absorption coefficientsof the material,Rd(r) gives the fraction of light thatexits pointxo given an impulse of incoming light ata pointxi that is distancer away. The total outgo-ing reflectance atxo in the direction−→wo is given byEquation 4. HereFt,i is the Fresnel transmittance atthe incident point andFt,o is the Fresnel transmit-tance at the exitant point.

Lo(xo,−→wo) =1π

Ft,o(η ,−→wo)∫

ARd(‖xo−xi‖) (4)∫

2π

Li(xi ,−→wi )Ft,i(η ,−→wi )(n·−→wi )dwidA(xi)

Measurement The σ ′s andσa of a homogeneous

material can be estimated by illuminating a smallpatch of with a thin beam of white light and fit-ting the resultant image values to Equation 3, asproposed by [6]. The DISCO method of [3] ex-tends this technique to measure the spatially varyingproperties of a smooth heterogeneous object. Theycapture impulse response images of the object tolaser light of different wavelengths for all positionsxi . This yields the functionRd(xi ,xo) for all pointsxi andxo on the surface, which is then used directlyto represent the spatially varying subsurface scat-tering properties. The very large acquisition times,however, make the technique unfeasible for humansubjects. In contrast, [10] use a projector and cam-era setup to project a series of images of point gridson the subject, reducing the acquisition time to acouple of hours. [15] also aim to capture the spa-tially varying properties of heterogeneous objects,but their focus is on materials with large mesostruc-tures. They factor light transport in such materialsinto a constant diffuse scattering term and spatiallyvarying mesostructure entrance and exit functions.[17] capture the translucency of skin using a spe-cialized measurement probe, and derive an averagevalue over a sparse sampling of points on severalfaces. This value can then be used to estimate a sin-gle set of scattering and absorption parameters for atarget face, given its average diffuse reflectance (seeSection 5). Spatially varying diffuse reflectance anda fixedσ ′

s have been used by [5] to derive spatiallyvarying σa over a face. In contrast to these tech-niques, our techniquequicklycaptures adensesam-pling of spatially-varying absorptionandscatteringparameters across the entire surface of a target face,using the same camera and projectors usually al-ready present in a typical 3D scanning setup. Inaddition to being used to render spatially varyingsub surface scattering, these parameters can be usedto provide a robust average of the scattering para-meters for a face. As shown in Figure 7b, usingthe average of our estimated parameters provides amore accurate result compared with using parame-ters derived from pre-estimated translucency values(in this case we are using the values from [17]).

Reflectance Fields A related concept to the8D BSSRDF is the 8D reflectance field [1],R(xi ,−→wi ;xr ,−→wr ), which is a generalization of theBSSRDF to an arbitrary closed surface,A. Thereflectance field represents the radiant exitant lightfield, Rr (xr ,−→wr ), from A in response to every possi-

ble incident light field,Ri(xi ,−→wi ).Image-based relighting methods attempt to cap-

ture the reflectance field of an object and renderfrom this data. They are capable of producing ex-tremely realistic images, since they inherently en-code all the complicated reflectance properties ofthe scene. [1] capture a subset of the reflectancefield of a human face by taking basis images of thesubject under directional illumination generated bya movable light source some distance away. An im-age of the face under any incident field of direc-tional illumination can be approximated by a linearcombination of these basis images. However, therestriction of illumination to distant light preventsthem from being able to simulate the effects of spa-tially varying light across the face.

[8] use a camera and projector setup to capture a6D reflectance fieldR(xi ,−→wi ;xr ), which can be usedto relight the scene with local illumination. Thecoarse resolution and large acquisition time, how-ever, make this technique inappropriate to apply tocapturing faces. [12] use a similar approach butgreatly reduce the number of images required, andconsequently the acquisition time, by using adap-tive refinement and Helmholtz reciprocity. Adap-tive refinement, however, requires online computeranalysis during capture, which currently cannot beperformed in real time.

3 Choice of SSS Parameters

In this work we fit our observed data to the di-pole diffusion model in order to estimate the spa-tially varying subsurface scattering properties of theface. Our method of using a lookup table with pre-calculated profiles is, however, independent of themodel used. It can just as easily be used to fit othermodels, such as the Sum of Gaussians sometimesused in real time rendering. Given the wide useof the dipole method, we use this model in orderto show that its parameters can quickly and eas-ily be estimated. Furthermore, the model expressesthe data in a compact representation,with just twoparameters per point, which can be represented asmaps over the face that are intuitive and easy to edit.These maps are a logical extension of existing mapssuch as specular and diffuse maps.

An alternative to fitting the desired model to thedata using a lookup table is to estimate the radialscattering profileRd(r) directly, without makingany assumptions about the scattering model. Ourdecision to forego this approach was based on a

couple of drawbacks. The problem of trying to re-cover the functionRd(r) at a pointx from imagesof x under multiple beams of light (our projectedpatterns) can be stated as recovering the impulseresponse of a system whose input and output areknown, and falls under the area of signal deconvo-lution. In general this problem is ill-conditioned,and the solutions, especially for impulse responsesthat are Gaussian-like in shape, are highly sensitiveto measurement noise. This is partly because thepower spectrum of these functions quickly becomesvery small, which leads to issues with numericalstability. By fitting the observed data to a modelinstead, we reduce the number of unknowns in thesystem, which makes a robust estimation possibleeven with noisy data.

4 Illumination Patterns

The optical parameters,σ ′s and σa, of a homoge-

neous translucent material can be accurately deter-mined by illuminating the surface with a thin beamof light and fitting Equation 3 to a one dimensionalradial profile of the spread [6]. Extending this ap-proach to capture spatially varying parameters ne-cessitates capturing prohibitively many images forreal time capture, [3, 10]. In order to reduce thenumber of images needed we make three assump-tions about the material: that it has slowly varyingscattering properties, that the material is opticallydense, and that scattering is isotropic [17]. Theassumption of an optically dense material impliesthat its behavior is dominated by multiple scatter-ing, and hence its properties can be faithfully mod-eled by the diffusion approximation. Furthermore,an optically dense material will have a short meanfree path and hence the scattering properties of thematerial in one part will only affect a small neigh-boring area.

To estimate per-point optical properties we needto capture images of the face under a small sequenceof light patterns. Ideally, each image should yieldscattering information for many parts on the facesimultaneously. To meet this goal, instead of mea-suring the radial decrease in intensity away from athin spot of light [6, 10], we measure the decreasein intensity away from afinite-widthstripe of lightthat is projected onto the face. Furthermore, usingthe assumption that light does not travel far withinthe skin, we illuminate the skin with multiple stripesof light simultaneously. Hence, our patterns are pe-riodic stripes of light that are swept across the sub-

Figure 1: (a) Face lit by the fully on projector, (b) oneof 40 phase-shifted stripe patterns, (c) after correcting forlambertian light attenuation, (d) estimate of indirect lightfrom subtracting the minimum of each pixel’s profile.

ject. Figure 1(b) is an image of one such pattern.The widths of the stripes and the distances be-

tween them are dictated by conflicting require-ments. Firstly, to minimize the interference ofneighboring stripes, so that points that fall in themiddle of two stripes are completely dark, thestripes need to be as far apart as possible. How-ever, to reduce the number of patterns, and hencethe time needed to capture all the images, we needto keep the stripes close together. Secondly, thewidths of the stripes should ideally be a single pro-jector pixel. Practically, however, at low light levelsthe subsurface scattering signal that we are tryingto observe is too low to be accurately distinguish-able from the camera noise. Hence, to increase thesignal to noise ratio we keep the stripes several pix-els thick. We found that 8 pixel wide stripes thatare 32 pixels apart, for a total of 40 patterns, is agood compromise between all these requirements.The approach presented in [11], which is based onHadamard coding, could also be used to generatesingle pixel width stripes with a better signal tonoise ratio. However, we found the informationavailable in our data was more than sufficient forestimating the scattering parameters of interest.

5 Parameter Estimation

We use the images of a subject under the projectedstripe patterns to createscattering profilesfor eachpixel. A pixel’s scattering profile is a 1D arrayof values representing its reflectance response overtime to the shifting patterns. Figure 2 shows thegraphs of four such profiles (with their phase shiftedso that the peak is centered), that were created fromour 40 input stripe patterns.

The per-pixel scattering and absorption parame-ters are estimated from these profiles by fitting theprofiles to the dipole model using a lookup table,similar to the specular lobe parameter estimation

Figure 2: Scattering Profiles of four different points onthe face for the three color channels, used for estimatingtranslucency. Note that the lips (c) have a comparativelyhigher profile in the red channel and the profiles in thestubble region (d) tend to zero more quickly (indicatingcomparatively lower translucency in all channels).

of [2]. We could also use a least squares fit tothe diffusion equation to estimate these parameters,but we found the LUT based approach to be faster.Our LUT contains pre-computed profiles indexedby two optical parameters, and we match a giveninput profile against the table to estimate its para-meters. Instead of usingσ ′

s andσa, we use a dif-ferent parametrization of the dipole model as pre-sented in [7], the total diffuse reflectance,Rd, andthe translucency, denoted by the diffuse mean freepath, ld. Given values ofRd and ld, [7] calculatevalues ofσ ′

s andσa by inverting Equations 5 and 6.

Rd =α ′

2

(1+e−

43 A√

3(1−α ′))

e−√

3(1−α ′)(5)

ld =1

σ ′t

√3(1−α ′)

(6)

The advantage of using this re-parametrization isthat the total diffuse reflectance valueRd, which isthe outgoing radiance of a point when all pointsaround it are illuminated, is available in a fully litimage of the subject after subtracting specular re-flection and indirect light (Figure 1(a)). Hence al-though we build a two-dimensional LUT ofRd vsld, for any given pixel we only have to perform aone-dimensional search for the value ofld, in therow given by itsRd value.

We estimate the total outgoing radiance at pixelxo resulting fromp nearby illuminated pixels usingthe simplified Equation 7. This equation ignores

Fresnel effects (since these effects are more im-portant at grazing angles where our data is unre-liable) and includes the cosine fall-off term insideLi(xi ,wi). Rd

o(r) is calculated using Equation 3 andthe properties atxo, andb is the constant amount ofindirect light falling on the pixel (section 5.2).

Lo(xo) = b+p

∑i=0

Rdo(r)Li(xi ,−→wi ) (7)

The time profiles in the LUT are constructed bycalculating for each stripe pattern, a pixelxo’s re-sponse to it as given by Equation 7.p are all pixelsin the camera image that get direct light from theprojector pattern and are also near enough toxo toinfluence its appearance. The calculation ofp de-pends on the ratio of projector distances to cameradistances, which in turn depends on the angle be-tween the projector and the surface point. In thiswork we assume that the surface is normal to thedirection of the projector. For more accurate resultswe could either capture the surface from additionalangles, or use a 3D table that takes into account theangle of the surface.

We use three different LUTs, one for each colorchannel, to maximize the resolution of our esti-mated parameters. Each LUT has a resolution of200×200, but covers different areas in the translu-cency and reflectivity spectra. Each location inthe LUT contains an area-normalized time profile,and the scattering and absorption parameters corre-sponding to theRd andld values. Four time profilesfrom different parts of the LUT are shown in Figure3 (d)-(g).

5.1 Parameter Profile Matching

The Rd value of a pixel is estimated by using thepixel’s value in the fully lit image, after the in-coming light has been normalized to one (Section7), and angular light fall off has been accountedfor (Section 6). We use cross-polarized illumina-tion to ensure that the observed reflectance valuesdo not contain a specular component. To deter-mine a pixel’sld value we use itsRd value to in-dex into the LUT and perform a one-dimensionalsearch in the corresponding row for the profile thatbest matches the area-normalized scattering profileof the pixel. Our error metric is the sum of squareddifferences between the two profiles, and since thismetric varies smoothly across the table for a giveninput profile, our search is implemented using a par-abolic minimum finder. Theσa andσ ′

s correspond-

(a) Red channel (b) Green channel (c) Blue channel

(d) (e) (f) (g)

Figure 3:Visual representation of the number of matchesfound for each entry of the LUT for the three color chan-nels (a,b,c) for the female subject. We see that red has ahigher average mean free path, and also that it has largevariations across the face compared to the other two chan-nels. Figures (d)-(g) show the area-normalized profiles forfour combinations ofRd andLd shown in (b).

ing to the best matching profile are then stored inseparate scattering and absorption maps.

Figure 6 shows some of the different maps thatwe estimate. We can see that the beard region hashigher absorption and lower translucency values,which was also visible in Figure 4. The lips have ahigher translucency, and significantly lesser absorp-tion in the green and blue channels. Figures 3 (a-c)shows the number of times a LUT entry is matchedto an input profile for the three color channels, in-tuitively showing the distribution of the parametersRd andld over the face.

5.2 Indirect Light Compensation

In areas of concavities the reflectance profiles thatwe observe may contain significant amounts of in-terreflected light. The contribution of this lighttransport has to be removed before we fit the subsur-face scattering model to the reflectance values. Wenote that most indirect light on a point generally ar-rives from areas that are relatively far and relativelylarge compared to the stripe distance: for example,indirect light on a point on the side of the nose ar-rives from all over the cheek. Thus it is a reasonableassumption that, given the frequency of our stripes,the same amount of indirect light arrives at a par-ticular point on the face regardless of which stripepattern phase is being projected. The same observa-tion was made earlier by [14] in regard to Gray codestripe patterns. This assumption is violated in areaswith local geometric detail (such as skin folds), andwe do not expect this technique to produce accurate

results in such regions.To find the amount of indirect light we first es-

timate the best matching curve as outlined in sec-tion 5.1, and then subtract from the minimum ofthe input curve the minimum of the matched curve.This gives us the amount of reflectance that the scat-tering model could not account for, most probablydue to the interreflected light. Following [14], weuse subtraction to compensate for this indirect light;we subtract its estimated value from the input pro-file and then rematch the profile to the LUT. Figure1(d) shows the amount of indirect light that we esti-mated for a particular capture. Note that the greatestamount of indirect light was calculated to be in con-cavities such as the eye sockets and under the chin.More recently, [9] uses a similar technique to esti-mate the direct and indirect light in various scenes.

6 Geometry Acquisition and Use

Since the surface of the face is non-planar, it isimportant to acquire geometry for both parameterestimation and rendering. For parameter estima-tion we have to correct for the decrease in lightintensity falling on a point as the angle betweenits normal and the light vector increases (see Fig-ure 1(a)). Failing to correct for this light attenua-tion would lead us to erroneously conclude that theskin is darker in such areas. Figure 1(b) shows howthe fully lit image looks after correcting for the co-sine light falloff, by dividing the reflectance valueat each pixel by the cosine of the angle between thelight and the normal vector. This correction is per-formed on each of our input images.

We estimate the face geometry by first derivinga sub-pixel accurate projector-camera correspon-dence image using the method described in [14],which reliably distinguishes between illuminationedges in the presence of subsurface scattering. Wethen triangulate the correspondence image using thecalibration information of the camera and projector,[18]. Finally, the geometry is smoothed to reducehigh-frequency extraction artifacts.

7 Results and Discussion

Setup. Our setup consists of a projector and cam-era pair aimed at the subject. We use a high res-olution camera (2352×1726 Basler A404) to cap-ture detailed images of the skin in a short amount oftime. The projector and camera are synchronizedby assigning alternating colors to a small square

σa [mm−1] σ ′s [mm−1] Translucency

Red Green Blue Red Green Blue Red Green Blue

female lips 0.0657 0.2505 0.2835 1.102 0.9084 0.7982 2.0844 1.0716 1.0426

forehead 0.0836 0.1774 0.2554 1.3079 1.5042 1.1741 1.6928 1.0571 0.9555

cheek 0.0525 0.1653 0.2216 1.2814 1.5000 1.1248 2.1817 1.1004 1.0569

mean 0.075 0.156 0.2254 1.3881 1.5017 1.0969 1.9282 1.2048 1.0988

std. dev. 0.0532 0.0652 0.0703 0.3484 0.3737 0.2496 0.3378 0.1986 0.1524

male lips 0.3227 0.6631 0.6833 0.7819 0.6710 0.5209 0.9670 0.6139 0.6365

forehead 0.4761 0.3163 0.442 1.1415 1.2267 0.8851 0.6578 0.8264 0.7538

stubble 0.4056 0.6024 0.6707 0.9237 0.8621 0.6648 0.7863 0.6147 0.6100

mean 0.2047 0.3595 0.4691 1.1602 1.1586 0.8373 1.1926 0.8164 0.7606

std. dev. 0.0989 0.1148 0.1179 0.1823 0.1985 0.1354 0.3127 0.15611 0.1325

Weyrich 06 1.8155 1.0213 0.6453

Jensen 01 skin1 0.032 0.17 0.48 0.74 0.88 1.01 3.6733 1.3665 0.6827

skin2 0.013 0.070 0.145 1.09 1.59 1.79 4.8215 1.6937 1.0899

Table 2: Estimated parameters for different areas on two different subjects, female and male. The last row is providedfor comparison against constant parameters estimated in previous work, [17, 6].

in the projector images. A photoreceptor detectsthese changes and triggers the camera. The setupis capable of displaying and capturing five patternsper second, with two images of different exposurescaptured for each projected pattern. The differentlyexposed images are compiled into HDR images af-ter subtracting the camera and projector black level.We calibrate the incident illumination by taking animage of a reflectance standard, and we divide allcaptured images by its average value. Finally, wecross-polarize the camera and the projector to elim-inate the specularly reflected light, which is neces-sary for correctly fitting our scattering model to theobserved values. The polarizers are placed diag-onally on the LCD projector and camera to avoidcolor shifts. We project a total of 88 images: 40stripe patterns, and 48 structured light images.

Table 2 shows a comparison of the estimatedparameters across the face for different subjects.There is a significant difference in both parametersacross the male and female subject; the male sub-ject’s skin exhibits higher absorption and smallerscattering. Our parameters are on the same scale asthose previously estimated, although the color dis-tribution is slightly different. In particular, [6] pre-dicts a greaterσ ′

s for the blue channel than the redchannel. This might be because of different colorresponse of our systems, or because [6] obtain mea-surements on the subject’s arm instead of the face.

We validate our parameters with renderings offaces produced under different incident illuminationand we compare them to ground truth images (cap-

tured under cross-polarized illumination to elimi-nate specular reflections). Our renderings do notreproduce the complete reflectance of a person’sface, as we ignore the specular surface reflectionand interreflected light, which would be simulatedin the context of a complete face scanning system(e.g. [1, 17]). Using the spatially varying propertiesestimated in Section 5 and the geometry estimatedin Section 6, we render the subsurface response ofthe face using the method outlined in [6]. We per-form a color correction step on our renderings usinga color matrix to compensate for crosstalk betweenthe color channels of the projector and the camera.

Figure 4 shows a comparison of our approachwith an image rendered with constant translucency(an average of the estimated translucency valuesover the entire face), and spatially varying reflec-tivity. The presence of whiskers inside the skinon the right side of the image reduces its translu-cency, an effect captured by our technique (Figure4(c)). Note that just a constant translucency para-meter (Figure 4(a)) cannot account for this. Figure7 shows that constant parameters also cannot cor-rectly predict the response of skin in the lip region.However, using constant parameters that are the av-erage of our estimated spatially varying parametersfor the subject, 7(b), provides a more accurate im-age compared with parameters derived from previ-ously published translucency values 7(a). Figure 5shows a comparison between a synthetic renderingcreated using our technique and a real image cap-tured under similar light. Our technique correctly

renders the scattering of light across the illumina-tion edge and the skin color in fully lit areas.

8 Limitations and Future Work

There are several limitations to our method that sug-gest logical directions for future work. Since wecapture images from a single viewpoint, both ourgeometry and estimated parameters are unreliablein areas where the angle between the surface normaland the projector or camera is large (e.g. the whiteareas in Figures 6a and 6b), or where they are notin focus. In practice the acquisition should be re-peated for multiple viewpoints, e.g. left, front, andright, and the results merged so that the most fronto-parallel data is used from each position. Alterna-tively, we could create a 3D LUT, where the thirddimension represents the angle between the normalat a point and the projector direction. Since theseangles are already known for each point, the 3D ta-ble would not impose any extra cost for searching.Although our LUT-based approach to estimationis extremely fast, it assumes locally flat geometry.This assumption is valid for most image regions butcauses miscalculation of parameters for points neargeometric discontinuities. For such places, a moreaccurate but more expensive approach would be touse the geometric information and a least squaressolution. A good compromise, however, is to man-ually correct such errors with information from theneighboring area, which is reasonable since the pa-rameter maps that we estimate are images that caneasily be edited and touched up.

9 Conclusion

We have presented an image-based acquisition sys-tem that quickly and efficiently estimates the scat-tering properties of an optically dense translucentnon-homogeneous material. Our system allows usto capture the spatially varying properties of skin ata high resolution, which was not possible with ear-lier approaches because of their large time require-ments. Our method can also be used to estimateimproved average scattering parameters for the skinmore conveniently and efficiently than earlier point-sampling based approaches. The technique is min-imally invasive, using neither potentially danger-ous lasers [6] or specialized equipment [17]. Ourdata representation is compact and editable, withjust two floating-point values per pixel, and can beused to render realistic faces under local illumina-

tion. Our data capture approach is a logical additionto existing face-scanning approaches e.g. [1, 16, 17]as it requires only a projector and camera.

References[1] Paul Debevec, Tim Hawkins, Chris Tchou, Haarm-Pieter

Duiker, Westley Sarokin, and Mark Sagar. Acquiring thereflectance field of a human face. InSIGGRAPH, 2000.

[2] Andrew Gardner, Chris Tchou, Tim Hawkins, and Paul De-bevec. Linear light source reflectometry.ACM Transactionson Graphics, 22(3):749–758, July 2003.

[3] M. Goesele, H. P. A. Lensch, J. Lang, C. Fuchs, and H.-P. Seidel. Disco: acquisition of translucent objects.ACMTransactions on Graphics, 23(3):835–844, August 2004.

[4] Pat Hanrahan and Wolfgang Krueger. Reflection from lay-ered surfaces due to subsurface scattering. InProceedings ofSIGGRAPH 93, August 1993.

[5] Christophe Hery. Implementing a skin bssrdf. RenderMan:Theory and Practice, Siggraph Course Notes, 2003.

[6] H. W. Jensen, S. R. Marschner, M. Levoy, and P. Hanrahan.A practical model for subsurface light transport. InProceed-ings of ACM SIGGRAPH 2001, August 2001.

[7] Henrik Wann Jensen and Juan Buhler. A rapid hierarchicalrendering technique for translucent materials.ACM Trans-actions on Graphics, 21(3):576–581, July 2002.

[8] Vincent Masselus, Pieter Peers, Philip Dutre, and Yves D.Willems. Relighting with 4d incident light fields.ACMTransactions on Graphics, 22(3):613–620, July 2003.

[9] S.K. Nayar, G. Krishnan, M. D. Grossberg, and R. Raskar.Fast Separation of Direct and Global Components of a Sceneusing High Frequency Illumination. InSIGGRAPH, 2006.

[10] Pieter Peers, Karl vom Berge, Wojciech Matusik, Ravi Ra-mamoorthi, Jason Lawrence, Szymon Rusinkiewicz, andPhilip Dutre. A compact factored representation of hetero-geneous subsurface scattering. InSIGGRAPH, 2006.

[11] Yoav Y. Schechner, Shree K. Nayar, and Peter N. Belhumeur.A theory of multiplexed illumination. In9th IEEE Interna-tional Conference on Computer Vision (ICCV’03).

[12] Pradeep Sen, Billy Chen, Gaurav Garg, Stephen R.Marschner, Mark Horowitz, Marc Levoy, and Hendrik P. A.Lensch. Dual photography. InSIGGRAPH, 2005.

[13] J. Stam. Multiple scattering as a diffusion process. InEuro-graphics Rendering Workshop 1995, 1995.

[14] Christopher D. Tchou. Image-based models: Geom-etry and reflectance acquisition systems. Master’sthesis, University of California at Berkeley, 2002,http://gl.ict.usc.edu/research/3DScanning/Tchou-Thesis-2002.pdf.

[15] Xin Tong, Jiaping Wang, Steve Lin, Baining Guo, andHeung-Yeung Shum. Modeling and rendering of quasi-homogeneous materials. InSIGGRAPH, 2005.

[16] Andreas Wenger, Andrew Gardner, Chris Tchou, JonasUnger, Tim Hawkins, and Paul Debevec. Perfor-mance relighting and reflectance transformation with time-multiplexed illumination. InSIGGRAPH, 2005.

[17] T. Weyrich, W. Matusik, H. Pfister, J. Lee, A. Ngan, H. W.Jensen, and M. Gross. Analysis of human faces using ameasurement-based skin reflectance model.ACM Transac-tions on Graphics, 25(3):1013–1024, July 2006.

[18] Zhengyou Zhang. A flexible new technique for camera cali-bration.IEEE Transactions on Pattern Analysis and MachineIntelligence, 22(11):1330–1334, 2000.

Figure 4: An image of a stripe of light projected onto a male subject’s neck (left), and stubble (right) area. Note thatthe real image (b), exhibits significantly less subsurface scattering in the stubble region as compared to the neck. Ourapproach (c), using spatially varying scattering parameters captures this effect, whereas the rendering using constantaverage translucency (a) overestimates the scattering in the stubble region and underestimates the scattering in the neck.

(a) Weyrich06 (b) average translucency (c) spatially varying scattering (d) real image

Figure 7: A curved stripe of light projected over the female subject’s lips. (a) and (b) are rendered with constanttranslucency; parameters from [17] and an average of our estimated parameters, respectively. Note that the scatteringin (b) has a yellower tone than the real image. (c) is rendered using spatially varying parameters, and comes closest tocapturing the color and influence of the subsurface transport in the real image, (d).

(a) real image

(b) synthetic rendering

Figure 5:Comparison of a rendering using our estimatedparameters with a real image under spatially-varying inci-dent illumination (high exposure on left, low exposure onright). The amount of scattering along the shadow edge isgenerally consistent between the two, except for the inter-reflected light which is not simulated.

Figure 6:From the top: recovered translucency, absorp-tion σa, and scatteringσ ′

s maps for the three color chan-nels. The white parts in (a,d) are areas where we cannotestimate good parameters, see Section 8.