PRT Summary

Motivation for Precomputed Transfer

• better light integration and light transport – dynamic, area lights – shadowing – interreflections

• in real-time

point lightpoint light area lightarea light

area lighting,area lighting,no shadowsno shadows

area lighting,area lighting,shadowsshadows

Precomputed Radiance Transfer (PRT)

represent lighting using spherical harmonics low frequency = few coefficients

n=4n=4 n=9n=9 n=25n=25

n=676n=676 originaloriginal

Tabulating PRT

LightinLighting Basis g Basis

ii

LightingLightingBasis Basis i+1i+1

LightingLightingBasis Basis i+2i+2

illuminateilluminate store store coefs on coefs on surfacesurface

......

......

PRT is a linear operator (matrix) per PRT is a linear operator (matrix) per surface point maps incident lighting to surface point maps incident lighting to

exit radianceexit radiance

Self-Transfer Results (Diffuse)

No Shadows/Inter Shadows Shadows+InterNo Shadows/Inter Shadows Shadows+Inter

Self-Transfer Results (Glossy)

No Shadows/Inter Shadows Shadows+InterNo Shadows/Inter Shadows Shadows+Inter

PRT Terminology

PRT Terminology

PRT Terminology

PRT Terminology

( ) ( ) ( )T Tp p p p p pe v v v y e y M l

p pe M l

PRT as a Linear Operator

• l : light vector (in source basis)• Mp : source-to-exit transfer matrix• ep : exit radiance vector (in exit basis)• y(vp) : exit basis evaluated in direction vp

• ep(vp) : exit radiance in direction vp

Tp pe y M l

PRT Special Case: Diffuse Objects

transfer vector rather than matrix

• independent of view (constant exit basis)• matrix is row vector• previous work uses different light bases• image relighting

[PRT02] SH[Xi03] Directional[Ng03] Haar[Ashikhmin02] Steerable

( ) ( )Tp p p pe v v y M l

PRT Special Case: Surface Light Fields

transfer vector rather than matrix

• frozen lighting environment• matrix is column vector

[Miller98][Nishino99][Wood00][Chen02][Matusik02]

Factoring PRT (BRDFs)

( ) ( )

p p p

Tp p p pe v v

e B R T l

y e

• Tp: source → transferred incident radiance

• Rp : rotate to local frame

• B: integrate against BRDF [Westin92]

• y(vp) • ep: evaluate exit radiance at vp

Hemispherical Projection

• exit radiance is defined over hemisphere, not sphere• spherical harmonics not orthogonal over hemisphere• how to project hemispherical functions using SH?

– naïve projection assumes “underside” is zero– least squares projection minimizes approximation error

• see appendix

Factoring PRT (BRDFs)

Technique LightB ExitB Note

[Sloan02] SH SH Phong

[Kautz02] SH Dir Arb

[Lehtinen03] SH Dir Lsq

[Matusik02] Dir Dir IBR

( ) ( )Tp p p p pe v v y B R T l

Extending PRT to BSSRDFs

• already handled by original equation• use [Jensen02], only multiple scattering (matrix

with only 1 row)• mix with “conventional” BRDF

( ) ( )Tp p p p

p p p p

e v v

y M l

M S B R T

Problems With PRT

• Big matrices at each surface point– 25-vectors for diffuse, x3 for spectral

– 25x25-matrices for glossy

– at ~50,000 vertices

• Slows glossy rendering (4hz)– Frozen View/Light can increase performance

– Not as GPU friendly

• Limits diffuse lighting order– Only very soft shadows

Compression Goals

• Decode efficiently– As much on the GPU as possible– Render compressed representation directly

• Increase rendering performance– Make non-diffuse case practical

• Reduce memory consumption– Not just on disk

Compression Example

Surface is curve, signal is normal

Compression Example

Signal Space

VQ

Cluster normals

VQ

Replace samples with cluster mean

pp p C M M M

PCA

Replace samples with mean + linear combination

0

1

Ni i

p p pi

w

M M M M

0

1p p

Ni i

p p C p Ci

w

M M M M

CPCA

Compute a linear subspace in each cluster

CPCA

• Clusters with low dimensional affine models• How should clustering be done?

• Static PCA– VQ, followed by one-time per-cluster PCA– optimizes for piecewise-constant reconstruction

• Iterative PCA– PCA in the inner loop, slower to compute– optimizes for piecewise-affine reconstruction

Static vs. Iterative

Related Work

• VQ+PCA [Kambhatla94] (static)• VQPCA [Khambhatla97] (iterative)• Mixture PC [Dony95] (iterative)• More sophisticated models exist

– [Brand03], [Roweis02]– Mapping to current GPUs is challenging

• Variable storage per vertex• Partitioning is more difficult (or requires more

passes)

Equal Rendering Cost

VQVQ PCAPCA CPCACPCA

0

1

( ) ( )p p

NT i i

p p p C p Ci

e v v w

y M l M l p

iCe 0

pCe

Rendering with CPCA

( ) ( )Tp p p pe v v y M l

0

1p p

Ni i

p C p Ci

w

M M M

0

1

( ) ( )p p

NT i i

p p p C p Ci

e v v w

y M M l

0

1

( ) ( )p p

NT i i

p p p C p Ci

e v v w

y e e

Rendering with CPCA

Constant per cluster – precompute on the CPURendering is a dot productCompute linear combination of vectors

Only depends on # rows of M

Non-Local Viewer

Assume: • vp constant across object (distant viewer)

Rendering independent of view & light orders- linear combination of colors

0

1

( ) ( )p p

NT i i

p p p C p Ci

e v v w

y M l M l

0

1

( ) ( )p p

NT i T i

p g C p g Ci

e v w v

y M l y M l

Rendering

= + +

Overdraw

1

1

1

3

3

22

2 2

22

22

22

2

2

2

faces belong to 1-3 clusters

OD = 1 face drawn onceOD = 2 face drawn 2xOD = 3 face drawn 3x

coherence optimization:• reclassification• superclustering

PixelShader

TextureConstants

Eye PointXform Matrix

GPU Dataflow

Vertices

1

NormalTangent

InClusterp

N

p

C

w

w

Vertex Shader

0

p

Ni i

Ci

w

e e

local view vecv

1

1

1

0

1C

C

NC

ee

e

( )T vy e

( )T vy

Exit Rad.

Demo

Results

Model #Pts SPCA IPCA FPS

Buddha 49.9k 3m30s 1h51m 27

BuddhaSS 49.9k 6m12s 4h32m 27

Bird Anis 48.7k 6m34s 3h43m 45

Bird Diff 48.7k 43s 3m26s 227

Head 50k 4m20s 2h12m 58.5

All examples have 25x25 matrices, 256 clusters, 8 PCA vectors

Conclusions

CPCA• works in “signal space”, not “surface space”• uses affine subspace per-cluster• compresses PRT well• is used directly without “blowing out” signal• requires small, uniform state storage• provides

– faster rendering– higher-frequency lighting

Future Work

• time-dependent and parameterized geometry• higher-frequency lighting• combination with bi-scale rendering• better signal continuity

Questions?

• DirectX SDK for PRT available soon.

• Jason Mitchell, Hugues Hoppe, Jason Sandlin, David Kirk

• Stanford, MPI for models