Precomputed Radiance Transfer Precomputed Radiance Transfer for Real-Time Rendering in Dynamic, for Real-Time Rendering in Dynamic,

Low-Frequency Lighting EnvironmentsLow-Frequency Lighting Environments

Peter-Pike Sloan, Microsoft ResearchPeter-Pike Sloan, Microsoft Research

Jan Kautz, MPI InformatikJan Kautz, MPI Informatik

John Snyder, Microsoft ResearchJohn Snyder, Microsoft Research

Previous Work – Where We FitPrevious Work – Where We Fit

TransportTransportComplexityComplexity

LightingLighting

simplesimple shadowsshadows

pointpointlightslights

full full env.env.mapmap

inter-inter-reflectionsreflections

singlesingleareaarealightlight

????[Moeller02][Moeller02]

[Blinn76][Blinn76]

[Heidrich00][Heidrich00]

[Crow77][Crow77]

[Crow77][Crow77]

[Williams78][Williams78]

[Stamminger02][Stamminger02]

[Heidrich00][Heidrich00]

[Malzbender01][Malzbender01]

[Tong02][Tong02]

[Blinn76][Blinn76]

[Miller84][Miller84]

[Latta02][Latta02]

[Ramamoorthi02][Ramamoorthi02][Ashikhmin02][Ashikhmin02]

our our techniquetechnique

[Moeller02][Moeller02][Ashikhmin02][Ashikhmin02]Frozen LightingFrozen Lighting

Irradiance VolumesIrradiance Volumes[Greger96][Greger96]

Surface LightfieldsSurface Lightfields[Miller98,Wood00][Miller98,Wood00][Chen02][Chen02]

Non-InteractiveNon-Interactive

[Matusik02][Matusik02]

MotivationMotivation

• Better light integration Better light integration and transportand transport dynamic, area lightsdynamic, area lights self-shadowingself-shadowing interreflectionsinterreflections

• For diffuse and For diffuse and glossy surfacesglossy surfaces

• At real-time ratesAt real-time rates

point lightpoint light area lightarea light

area lighting,area lighting,no shadowsno shadows

area lighting,area lighting,shadowsshadows

Basic IdeaBasic Idea

( )V s

( ) ( )i iL s l B s

( ) ( ) ( ) ( , ) ( )i i NR v l B s V s f s v H s d s

( ) ( ) ( ) ( , ) ( )i i NR v l B s V s f s v H s d s

( ) max( ,0)NH s s N Preprocess for all Preprocess for all ii

v

( ) ( ) ( ) ( , ) ( )NR v L s V s f s v H s d s

( ) i iR v l t

lightlight

2D example, piecewise constant basis, shadows only2D example, piecewise constant basis, shadows only

2p2p

1p1p3p3p

Diffuse Self-TransferDiffuse Self-Transfer

PreprocessPreprocess

1p1p

2p2p

3p3p

Project LightProject Light

lightlight

RenderingRendering

•• ==1p1p

1p1p

2p2p

2p2p ==••

3p3p

3p3p ==••

PrecomputationPrecomputation

Basis 16Basis 16

Basis 17Basis 17

Basis 18Basis 18

illuminateilluminate resultresult

......

......

Previous Work Previous Work – – Scene RelightingScene Relighting

• [Dorsey91] opera lighting design[Dorsey91] opera lighting designadjusts intensity of fixed light sourcesadjusts intensity of fixed light sources

• [Nimeroff94] natural environments[Nimeroff94] natural environmentsuses steerable functions for general skylight illuminationuses steerable functions for general skylight illumination

• [Teo97] efficient linear re-rendering[Teo97] efficient linear re-renderinggeneralizes to non-infinite sources, PCA to reduce basisgeneralizes to non-infinite sources, PCA to reduce basis

• [Debevec00] reflectance field of a face[Debevec00] reflectance field of a faceuses directional light basis for relighting facesuses directional light basis for relighting faces

• [Dobashi95] lighting design[Dobashi95] lighting designuses SH basis for point light intensity distributionuses SH basis for point light intensity distribution

Basis FunctionsBasis Functions

• We use We use Spherical HarmonicsSpherical Harmonics

• SH have nice properties:SH have nice properties: simple projection/reconstructionsimple projection/reconstruction rotationally invariant (no aliasing)rotationally invariant (no aliasing) simple rotationsimple rotation simple convolutionsimple convolution few basis functions few basis functions low freqs low freqs

l=0 m=0l=0 m=0

l=1 m=-1l=1 m=-1

l=1 m=0l=1 m=0

l=1 m=1l=1 m=1

l=2 m=1l=2 m=1 l=3 m=-1l=3 m=-1 l=3 m=2l=3 m=2 l=4 m=-2l=4 m=-2

Diffuse Transfer ResultsDiffuse Transfer Results

No Shadows/Inter Shadows Shadows+InterNo Shadows/Inter Shadows Shadows+Inter

Glossy Self-TransferGlossy Self-Transfer

exiting radiance is view-dependentexiting radiance is view-dependent

depends on BRDF (we use Phong)depends on BRDF (we use Phong)

( ) ( ) ( ) ( , ) ( )NR v L s V s f s v H s d s

Glossy Self-TransferGlossy Self-Transfer

exiting radiance is view-dependentexiting radiance is view-dependent

depends on BRDF (we use Phong)depends on BRDF (we use Phong)

represent represent transferredtransferred incidentincident radiance, not exitingradiance, not exiting

accounts for shadows, interreflectionsaccounts for shadows, interreflections

allows run-time BRDF changesallows run-time BRDF changes

( ) ( ) ( ) ( , ) ( )NR v L s V s f s v H s d s

( ) ( ) ( , ) ( )tranNR v L s f s v H s d s

( ) ( )tranL s M L s

Transfer MatrixTransfer Matrix

Precompute how global lighting Precompute how global lighting transferred lighting transferred lighting

**

**

lig

hti

ng

lig

hti

ng

transfer matricestransfer matrices transferred radiancetransferred radiance

pp11

pp22

pp11

pp22

Glossy RenderingGlossy Rendering

Lighting Env in SH Lighting Env in SH (vector)(vector)

Transfer matrix (maps radiance to Transfer matrix (maps radiance to transferred incident radiance)transferred incident radiance)

Integrates incident radiance Integrates incident radiance against BRDF in direction vagainst BRDF in direction v

Phong: Convolve light and Phong: Convolve light and evaluate in reflection direction, Revaluate in reflection direction, R

Freeze the viewFreeze the viewFreeze the LightFreeze the Light

Tpv pv pr b M l

Lookup Lookup at Rat R

Glossy Transfer ResultsGlossy Transfer Results

No Shadows/Inter Shadows Shadows+InterNo Shadows/Inter Shadows Shadows+Inter

• Glossy object, 50K meshGlossy object, 50K mesh

• Runs at 3.6/16/125fps on Runs at 3.6/16/125fps on 2.2Ghz P4, ATI Radeon 85002.2Ghz P4, ATI Radeon 8500

Interreflections and CausticsInterreflections and Caustics

* * *( ( | ))LS S D G P

LP

*( | )L D G P

nonenone 1 bounce1 bounce 2 bounces2 bounces

causticscaustics

interreflectionsinterreflections

LGP

Transport PathsTransport Paths

Runtime is independent Runtime is independent

of transport complexityof transport complexity

Arbitrary BRDFs [Kautz02]Arbitrary BRDFs [Kautz02]

==T

pv pv p pr b R M l

( ) ( ) ( , ) ( )tranp Nr v L s f v s H s d s

( ) ( ( )) ( , ) ( )p i i Nr v l y s f v s H s d s

( ) ( ) ( , ) ( )p i i Nr v l y s f v s H s d s

……

BRDF CoefficientsBRDF Coefficients

Arbitrary BRDF ResultsArbitrary BRDF Results

Other BRDFsOther BRDFs Spatially VaryingSpatially VaryingAnisotropic BRDFsAnisotropic BRDFs

Neighborhood TransferNeighborhood Transfer

• Allows to cast shadows/caustics onto Allows to cast shadows/caustics onto arbitrary receiversarbitrary receivers

• Store how object scatters/blocks light Store how object scatters/blocks light around itself (transfer matrices on grid)around itself (transfer matrices on grid)

**

**

transfer matricestransfer matrices transferred radiancetransferred radiance

receiverreceiver

lig

hti

ng

lig

hti

ng

receiverreceiver

Neighborhood Transfer ResultsNeighborhood Transfer Results

• 64x64x8 neighborhood64x64x8 neighborhood

• diffuse receiverdiffuse receiver

• timings on 2.2Ghz P4, timings on 2.2Ghz P4, ATI Radeon 8500 ATI Radeon 8500

• 4fps if light changes4fps if light changes

• 120fps for constant light120fps for constant light

VolumesVolumes

• Diffuse volume: 32x32x32 gridDiffuse volume: 32x32x32 grid

• Runs at 40fps on 2.2Ghz P4, ATI 8500Runs at 40fps on 2.2Ghz P4, ATI 8500

• Here: dynamic lightingHere: dynamic lighting

Local Lighting using Radiance SamplingLocal Lighting using Radiance Sampling

• Sample incident radiance at Sample incident radiance at multiplemultiple points points

• Choose sample points over object using ICP from VQChoose sample points over object using ICP from VQ

• Correct for shadows but not interreflectionsCorrect for shadows but not interreflections

single samplesingle sample(at center = light at (at center = light at ))

multi-sample multi-sample resultresult

multi-sample multi-sample locationslocations

Light Size vs. SH OrderLight Size vs. SH Order

20°20°

40°40°

0°0°

n=2n=2linearlinear

n=3n=3quadraticquadratic

n=4n=4cubiccubic

n=5n=5quarticquartic

n=6n=6quinticquintic

n=26n=26 n=26n=26windowedwindowed

RTRT

ResultsResults

Live Demo (Radeon 9700)Live Demo (Radeon 9700)

ConclusionsConclusions

Contributions:Contributions:• Fast, arbitrary dynamic lightingFast, arbitrary dynamic lighting

on surfaces or in volumeson surfaces or in volumes

• Includes shadows and interreflectionsIncludes shadows and interreflections• Works for diffuse and glossy BRDFsWorks for diffuse and glossy BRDFs

Limitations:Limitations:• Works only for low-frequency lightingWorks only for low-frequency lighting• Rigid objects only, no deformationRigid objects only, no deformation

Future WorkFuture Work

• Practical glossy transferPractical glossy transfer Eliminate frozen view/light constraintsEliminate frozen view/light constraints Compress matrices/vectorsCompress matrices/vectors

• Enhanced preprocessingEnhanced preprocessing Subsurface scattering, dispersionSubsurface scattering, dispersion Simulator optimizationSimulator optimization Adaptive sampling of transfer over surfaceAdaptive sampling of transfer over surface

• Deformable objectsDeformable objects

AcknowledgementsAcknowledgements

• Thanks to:Thanks to: Jason Mitchell & Michael Doggett (ATI) Jason Mitchell & Michael Doggett (ATI) Matthew Papakipos (NVidia) Matthew Papakipos (NVidia) Paul Debevec for light probesPaul Debevec for light probes Stanford Graphics Lab for Buddha modelStanford Graphics Lab for Buddha model Michael Cohen, Chas Boyd, Hans-Peter Seidel for Michael Cohen, Chas Boyd, Hans-Peter Seidel for

early discussions and supportearly discussions and support

Questions?Questions?

PerformancePerformance

ModelModel # Verts# Verts GF4 4600GF4 4600

FPSFPSR300R300

FPSFPSPre-Pre-

computecompute

MaxMax 50,06050,060 215215 304304 1.1h1.1h

BuddhaBuddha 49,99049,990 191191 269269 2.5h2.5h

TweetyTweety 48,66848,668 240240 326326 1.2h1.2h

TyraTyra 100,000100,000 118118 179179 2.4h2.4h

TeapotTeapot 152,413152,413 9393 154154 4.4h4.4h

Matrix FormulationMatrix Formulation

( ) ( ) ( )ij i jM B s B s V s d s

( ) ( ) ( )i iM L s B s V s d s

( ) ( ) ( )i j j iM l B s B s V s d s

( ) ( ) ( )i j j iM l B s B s V s d s

Results – PreprocessingResults – Preprocessing

ModelModel TypeType SamplingSampling Preproc.Preproc. FPSFPS

headhead diffusediffuse 50K vert.50K vert. 1.1h1.1h 129129

ringring diffusediffuse 256x256 t.256x256 t. 8m8m 9494

buddhabuddha diffusediffuse 50K vert.50K vert. 2.5h2.5h 125125

buddhabuddha glossyglossy 50K vert.50K vert. 2.5h2.5h ..125..125

tyratyra diffusediffuse 100K vert.100K vert. 2.4h2.4h 8383

tyratyra glossyglossy 100K vert.100K vert. 2.4h2.4h ..83..83

teapotteapot glossyglossy 150K vert.150K vert. 4.4h4.4h ..49..49

cloudcloud diffusediffuse 32x32x3232x32x32 15m15m 4040

gliderglider neighb.neighb. 64x64x864x64x8 3h3h ..120..120

Previous Work – Precomputed TransportPrevious Work – Precomputed Transport

• [Greger96] irradiance volumes [Greger96] irradiance volumes move diffuse object through precomputed lightingmove diffuse object through precomputed lighting

• [Miller98,Wood00,Chen02] surface lightfields[Miller98,Wood00,Chen02] surface lightfieldsfrozen lighting environmentsfrozen lighting environments

• [Ashikmin02] steerable illumination textures[Ashikmin02] steerable illumination texturessteers small light source over diffuse objectsteers small light source over diffuse object

• [Matusik02] image-based 3D photography[Matusik02] image-based 3D photographysurface lightfield + reflectance field – not interactivesurface lightfield + reflectance field – not interactive

Dynamic LightingDynamic Lighting

• Sample incident lighting on-the-flySample incident lighting on-the-fly precompute textures for SH basis functions precompute textures for SH basis functions use cube map parameterizationuse cube map parameterization render into 6 cube map faces around render into 6 cube map faces around pp read images backread images back projection: dot-product between cube mapsprojection: dot-product between cube maps

• ResultsResults low-resolution cube maps sufficient: 6x16x16low-resolution cube maps sufficient: 6x16x16 average error: 0.2%, worst-case: 0.5% average error: 0.2%, worst-case: 0.5% takes 1.16 ms on P3-933Mhz, ATI 8500takes 1.16 ms on P3-933Mhz, ATI 8500

pLpL