all-frequency rendering of dynamic, spatially-varying reflectance
DESCRIPTION
All-Frequency Rendering of Dynamic, Spatially-Varying Reflectance. Jiaping Wang, Peiran Ren , Minmin Gong, John Snyder, Baining Guo SIGGRAPH Asia 2009. Presenter: Kevin April 14, 2010. Authors. Peiran Ren [2]. Minmin Gong [1]. John Snyder [3]. Baining Guo [2]. Jiaping Wang - PowerPoint PPT PresentationTRANSCRIPT
1
All-Frequency Rendering of Dynamic, Spatially-Varying Reflectance
Jiaping Wang, Peiran Ren, Minmin Gong,John Snyder, Baining Guo
SIGGRAPH Asia 2009
Presenter: KevinApril 14, 2010
2
Authors
Jiaping Wang [1]
Peiran Ren [2]
Minmin Gong [1]
John Snyder [3]
Baining Guo [2]
Microsoft Research Asia [1]
Tsinghua University [2]
Microsoft Research [3]
3
Introduction
4
Introduction
• Final goal of real-time realistic rendering– Dynamic lighting– Changeable viewpoint– All-frequency effects– Dynamic, editable, and spatially-varying materials– Dynamic, deformable scenes
Ng et al. SIGGRAPH ‘04 Wang et al. SIGGRAPH Asia ‘09 Sloan et al. SIGGRAPH ‘05
5
Introduction-2
• Challenges– BRDF complexity
• Modeling the complex reflectance of real world materials
– Light integration complexity• Integration over the whole hemisphere (cannot afford especially
when environment maps are used)
– Light transport complexity• Interreflection, shadows, …etc
6
Introduction-2
Off-lineHigh Quality Real-time
Real-timeRay-tracing
ProgrammableGraphics Hardware
PrecomputedRadiance Transfer
Wang et al. SIGGRAPH ‘09 Dachsbacher et al. SIGGRAPH ‘07Ritschel et al. SIGGRAPH Asia ‘08
7
Introduction-3
• Precomputed Radiance Transfer (PRT)– The term comes from [Sloan ’02]– Precompute “light transport function”
– Compress by basis (SH, Wavelet, SRBF…)– The computation of rendering equation reduces to
inner/vector products in the run time
iioiiio dxnxVLxL
)]0),(max(),()[,()(),(4
8
Introduction-4
• PRT timeline (02~05)
02 03 04 05
Sloan et al. SIGGRAPH ‘02
Ng et al. SIGGRAPH ‘03
Sloan et al. SIGGRAPH ‘03
Ng et al. SIGGRAPH ‘04
Wang et al. EGSR ‘04
Lui et al. EGSR ‘04
Sloan et al. SIGGRAPH ‘05
Zhou et al. SIGGRAPH ‘05
Pioneer, SH
CPCA
Wavelet
Triple Product
In-OutFac.
DynamicScenes
9
Introduction-5
• PRT timeline (06~09)
07 08 09
Wang et al. ACMTOG ‘06
Green et al. I3D ‘06 Green et al. EGSR ‘07
Sun et al. SIGGRAPH ‘07
Ben-Artzi et al. ACMTOG ‘08
Ramamoorthi CG&V ‘09
Wang et al. SIGGRAPH Asia ‘09
06
Xu et al. TVCG ‘08
Ben-Artzi et al. SIGGRAPH ‘06
Tsai et al. SIGGRAPH ‘06
SRBF
BRDF-Editing
CTA
SVBRDF, BRDF-Editing
SurveyBRDF-EditingWith G.I.
BRDF-EditingWith G.I.
10
Introduction-6
• Summary– Compression• PCA, Clustered PCA (CPCA), Clustered Tensor
Approximation (CTA) …
– Basis• Spherical harmonics (SH)• Wavelets• Zonal harmonics (ZH)• Spherical Radial Basis Function (SRBF)
11
Introduction-7
• Summary (cont.)– How to choose good basis for representation?• Can model all-frequency effects• Rotational invariant• Accuracy• Compact
Fit clamped cosine term to basisWang et al. SIGGRAPH Asia ‘09 – supplement materials
12
Introduction-8
• Spherical Gaussian (SG)– A type of SRBF, symmetric around a specific lobe
axis
– All advantages in the previous page– Inner product & cross product can be efficiently
computed
)1(),,;( pvepvG lobe axis
lobe sharpnesslobe amplitude
13
Introduction-9
• SG mixtures
n
iiiipvGvF
1
* ),,;()(
n
iiiipvGvF
1
* ),,;()( RRRotated version
Single Lobe Two Lobes Multiple Lobes
14
Overview
15
This Paper
• Real-time• Dynamic (editable, change with time),
spatially-varying BRDFs• All-frequency effects from both environmental
and local point lights
• Static scenes• No interreflection
17
Contributions
• Propose two new representations for BRDFs and Visibility to compact the size of data– SG mixtures for microfacet-based reflectance• Accurate and compact• Parametric BRDFs can be fit on-the-fly• Fast rotation, warping, and products in run time
– SSDFs for visibility• Ghost-free, per-pixel interpolation
• Dynamic local point lights
18
• Decoupling BRDF from visibility
Approach Overview
6D SVBRDF
Represent
4D NDF(Microfacet)
Visibility(binary)
SSDF(Spherical Signed Distance Functions)
Fit into SGs
Map PCA
Mixture of SGs(Spherical Gaussians)
Eigen-Vectors
19
SVBRDF Representation
20
SVBRDF Representation
• Microfacet Model
• Why use microfacet model?– General– Compact
oi
hioihoi
FGD
coscos4
),(),()(),(
Fresnel Term
Geometry Term
NormalDistribution
Function
21
SVBRDF Representation-2
• Reflectance representation using SGs– Parametric BRDFs (on-the-fly fitting)
– Example: Cook-Torrance Model
)()(),( hDiMio oremaining factor
(shadowing+Fresnel)NDF
Very Smooth High-Frequency Fit into SGs
))((
),(),()(
onin
ioSioFiM CTCT
o
)1,/2,;()( 2)/)(arccos( 2
mnhGehD mnh
22
SVBRDF Representation-3
ground truth single-lobe SG(this paper)
256-term 64-term 16-termBRDF factorization
Cook-Torrancem=0.1
Cook-Torrancem=0.045
23
SVBRDF Representation-4
ground truth 7-lobe SG(this paper)
256-term 64-termBRDF factorization
Ashikhmin-Shirleynu=8,nv=128
Ashikhmin-Shirleynu=25,nv=400
Ashikhmin-Shirleynu=75,nv=1200
24
SVBRDF Representation-5
• Accuracy– Parametric isotropic models
25
SVBRDF Representation-6
• Accuracy (cont.)– Parametric anisotropic models
26
SVBRDF Representation-7
• Reflectance representation using SGs– Measured BRDFs (preprocessing)• Using tabulated NDF [Wang et al. SIGGRAPH ‘08] and
shadowing factor S at each surface point• Compress shadowing function by PCA (8D)• Fit NDF with a small number of SGs
27
SVBRDF Representation-8fabricgreen
delrin steelphenolicyellow
albmbronze
violetacrylic
groundtruth
1SG
2SG
3SG
256-TermFac.
64-TermFac.
28
SVBRDF Representation-9
• Accuracy– Measured BRDFs
29
Visibility Representation
30
Visibility Representation
• Spatially-varying visibility is represented with Spherical signed distance function (SSDF)– Directly interpolate binary visibilities will produce
ghost effects
– SSDF maps binary visibility to continuous function
31
Visibility Representation-2
• SSDF mapping– Positive: visible; negative: occluded– Value: the angular distance to the nearest direction
t on the shadow boundary
)(iV;1)( if ),arccos( min
0)(
iVit
tV
.0)( if ),arccos( min 1)(
iVittV
32
Visibility Representation-3
)(' iV
Reconstructed Visibility
d-2
(i) if ,1
otherwise ,0
δ: elevation angle
Inner product / vector product of SGs and V’(i) can be efficiently evaluated in the run time!
33
Visibility Representation-3
• Compression– Using PCA
• PCA coefficients are stored as vertex attributes and Interpolated to each pixel during rasterization• Eigenvectors are encoded in multiple textures
Nv
j
Vjxjx wiViV
1,)()(
PCA coefficientsPCA eigenvectors
34
Visibility Representation-4
• Compression results
Ray-Traced Uncompressed SSDF SSDF/PCA 384 Terms
SSDF/PCA 144 Terms SSDF/PCA 48 Terms SSDF/PCA 16 Terms
35
Lighting Representation
36
Lighting Representation
• Local point lights– Approximated with a single-lobe SG– Yielding a spatially-varying radiance field
• Infinitely-distant light from direction I
)||||
),||||
2(,
||||;()(
22
21*
xl
s
xl
rf
xl
xliGiL ax
l: 3D light position
s: intensity
)),2(,;()( 21* srfIiGiL a
37
Lighting Representation-2
• Distant environmental lighting– Apply to diffuse component• Fit the environment map with a SG mixture • [Tsai. et al. SIGGRAPH 2006]
– Apply to specular component• Preconvolve environmental radiance with SG kernels• The run-time inner product is reduced to a MIPMAP
texture fetch• [Kautz et al. EGWR 2000], [McAllister et al. GH 2002],
[Green et al. I3D 2006]
38
Run-Time Rendering
39
Run-Time Rendering
• Per-vertex attributes– Position, texture coordinates, local coordinate
frame, PCA coefficient for SSDF
• BRDF parameters, tabulated SG lobes (for measured BRDFs), and PCA-compressed shadow factors are stored in textures
interploated across triangle mesh when passing from vertex shader to pixel shader
40
Run-Time Rendering-2
)()( oRkRkoR ssdd
2
),0max()()(Sd diniiViLR
2),0max()(),()()(
S ss diniiVioiLoR
170.1 ,133.2 ,),,;();(* ccccxx niGniC CosineApproximation
LightingApproximation
)(* iL
BRDFApproximation
VisibilityApproximation
)())();(( ** iViLniCR dxxd )())(),();(()( **
,* iLiVoiniCoR d
xxsxs
SGs for NDFSpherical Warp
)()( ** iWhD Multiplied remaining factor
)()();( ** iWiMoi os
PCA coefficients of SSDFs
Uncompressedon GPU
41
Experimental Results
42
Experimental Results
• Data size and precomputation time
43
Experimental Results-2
• Per-vertex vs. per-pixel
wireframe per-vertex shading per-pixel shading
44
Experimental Results-3
• Distant environmental light + nearby point light
45
Experimental Results-4
• Results for isotropic BRDFs
46
Experimental Results-5
• Results for anisotropic BRDFs
47
Experimental Results-6
• Video
48
Conclusion
• Solution for highly-specular, spatially varying, dynamic materials, natural lighting, changeable viewpoints realistic rendering– SG mixtures for microfacet-based reflectance• Accurate and compact• Fast rotation, warping, and products in run time
– SSDFs for visibility• Ghost-free, per-pixel interpolation• Allow sparse set of per-vertex visibility samples
49
End
Thanks for yourAttention
50
[Sloan et al. 02]
• Precomputed Radiance Transfer for Real-Time Rendering in Dynamic, Low-Frequency Lighting Environment– Sloan, Kautz, and Snyder– SIGGRAPH 2002
51
[Sloan et al. 02]-2
• Algorithm (diffuse)– Express lighting as SH
– Reflection Equation becomes
– Define transfer function and project to SH
– Rendering reduces to
iiiijj jo dnxVYxL
)0,max(),()()(4
)()(2)1*(
1 i
l
j jji YL
iiiijj dnxVYxT
)0,max(),()()(4
)()(1 ij jjo TxL
Transfer vector (diffuse)Transfer Matrix (glossy)
52
[Ng et al. 03]
• All-Frequency Shadows Using Non-linear Wavelet Lighting Approximation– Ng, Ramamoorthi, Hanrahan– SIGGRAPH 2003
53
[Ng et al. 03]-2
• Relighting: Matrix-Vector Multiplication
)0),(max())(,(),(),( xnxxVxT io
)(),()( ii io LxTxL
TLLo
=Lo Li
Column:Lighting direction
Row:Pixel or Vertex
54
[Ng et al. 03]-3
• The dimension of T is too large– 300000 x 25000 for a 512x512 image and 6 x 64 x
64 environment mapApproximate by basis fitting(How to preserve high-frequency shadow?)
Approach:Approximate lighting dynamically at run-time using wavelet
100~200 basis coefficients are enough to generate high-quality results (compared to 20000 with SH)
55
[Ng et al. 04]
• Triple Product Wavelet Integrals for All-Frequency Relighting– Ng, Ramamoorthi, Hanrahan– SIGGRAPH 2004
56
[Ng et al. 04]-2
• Changeable view makes a 6D function• Factor the BRDF and visibility, reduce 6D
function into two 4D functions– For each spatial location and outgoing direction,
store 3 functions in cubemaps (Light, Visibility, BRDF)
– Calculate triple-product in run time
j k l lkjjkl
iilikijj k l lkj
il ilolk ikkj ijjo
VLC
dVL
dxnxVLxB
)()()(
)()),(()()()(),(
4
4
57
[Ng et al. 04]-3
58
[Wang et al. 04]
• All-Frequency Relighting of Non-Diffuse Objects using Separable BRDF Approximation– Rui Wang, John Tran, and David Luebke– EGSR 2004
59
[Wang et al. 04]-2
• Factor the BRDF into terms which depend only on incident / outgoing angles
• We can then define and precompute a view-independent transport function
• Thus, rendering is reduced to
K
k ikokkoi gh1
)()(),(
)0),(max()(),(),( xngxVxT iikiik
i
K
k ikiokko dxTLhxB
14
),()()(),(
60
[Wang et al. 04]-3
• Comparison– Triple product• Pros: support true all-frequency effects• Cons: performance
– BRDF in-out factorization• Pros: speed and simple• Cons: k can not be large, make it only suit for glossy or
broad specular lobes
61
[Tsai et al. 06]
• All-Frequency Precomputed Radiance Transfer using Spherical Radial Basis Functions and Clustered Tensor Approximation– Yu-Ting Tsai and Zen-Chung Shih– SIGGRAPH 2006
62
[Tsai et al. 06]-2
63
[Ben Artzi et al. 06]
• Real-Time BRDF Editing in Complex Lighting– Ben-Artzi, Overbeck, Ramamoorthi– SIGGRAPH 2006
64
[Ben Artzi et al. 06]-2
• Fixed the scene, lighting, and viewpoint• Write the BRDF as an expansion in terms of
basis functions, instead of lighting
Encapsulated in a single 1D function or curve
J
j jjoiqoioiqoi bcf1
)(),()),((),(),( Fixed part Editable part
65
[Ben Artzi et al. 08]
• A Precomputed Polynomial Representation for Interactive BRDF Editing with Global Illumination– Ben-Artzi, Egan, and Ramamorthi– ACMTOG 2008