Download - Thank you for the introduction
Mahdi M. Bagher / Cyril Soler / Kartic Subr / Laurent Belcour / Nicolas HolzschuchMahdi M. Bagher / Cyril Soler / Kartic Subr / Laurent Belcour / Nicolas Holzschuch
Interactive rendering of Interactive rendering of acquired materials on dynamic geometryacquired materials on dynamic geometry
using bandwidth predictionusing bandwidth prediction
• Direct reflection under distant illumination with no visibility and global illumination effects
• Rendering an image is computationally expensive.
Shading
from
Wik
iped
ia
Illumination BRDF
Acquired Materials
• Using measured materials• photo-realistic rendering• large memory footprint
(33MB for a single isotropic BRDF)
• Not suitable for real-time rendering
Phong shading Measured “Color-Changing-Paint-3”BRDF measurement Gantry [Matusik2002]
Monte Carlo Sampling
• Monte Carlo approximation to the shading integral
• Importance sampling• less noisy
• Data driven reflectance• no analytical importance function• pre-computed Importance samples
Monte Carlo sampling
BRDF importance sampling
Sampling
• Dealing with two types of sampling• Reconstruction• sampling in image space• to render an image
• Integration• sampling in the space of incident directions • to shade each individual pixel
Illumination BRDF Cosine
The Problem
• Shading is slow…
• How to speed-up shading?
Case Study
• What difference various materials make in rendering an image?
Measured reflectance data from [Matusik2002]
Diffuse Materials
• Diffuse materials• shading varies slowly across the image (low frequencies)• many integration samples for each pixel (wide BRDF lobe)
Measured Teflon [Matusik2002] BRDF Lobe [BRDFLab]
Specular Materials
• Specular materials• shading varies quickly across the image (high frequencies) • few integration samples for each pixel (narrow BRDF lobe)
Measured color-Changing-Paint3 [Matusik2002] BRDF Lobe [BRDFLab]
The Problem
• Rendering is slow…• How to speed-up shading?
• Can we exploit the coherency between • Reconstruction samples…• Integration samples…
• …for adaptive sampling?• How to measure the coherency
between samples?
Intuition
• Study the frequencies in the image• As a function of what happens to the light• Traveling from the light source to the camera
The Traveling Light
• Light when traveling is affected by• Travel through free space• Object’s curvature• Reflectance• BRDF• Texture
• Occlusion
Our Contribution
• How to speed-up shading?By sparse sampling the image• How to combine the sparse samples?• Multi-resolution rendering and bi-lateral up-sampling
By adaptive sampling for integration
Our Contribution (Illustrated)
Sparse reconstruction samples Adaptive Integration samples
,
Final shaded image
Multi-resolution shading and bi-lateral up-sampling
∝ the average variance of the shading integrand∝ the local maximum variations in the image
Assumptions
• Direct reflection under distant illumination
• No visibility and global illumination effects
• Allow dynamic editable geometry
• Screen-space rendering in the context of deferred-shading
• Any shading model is possible
• Save more if shading cost is higher e.g. measured materials
• BRDF importance sampling
Normal
Dynamic Geometry Animation
QuickTime™ and a decompressor
are needed to see this picture.
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Adaptive sampling for integration
• Validation
• The rendering algorithm
• Results and comparisons
• Extension: MSAA
• Conclusion / Future work
Related Work GPU Rendering of Complex Materials
[Heidrich&Seidel1999]
[Ramamoorthi2002]
[Kautz2002]
[Claustres2007]
[Kautz&McCool1999]
[Latta&Kolb2002][McCool2001]
[Wang2009]
Related WorkMulti-Resolution Screen-Space Algorithms
[Nichols&Wyman2009] [Shopf2009][Nichols&Wyman2010]
[Soler2010]
[Nichols2010][Segovia2006]
Related WorkPre-Computed Transport
[Sloan2002]
[Ramamoorthi2009]
[Sun2007]
[Liu2004]
[Loos2011]
[Loos2012]
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Adaptive sampling for integration
• Validation
• The rendering algorithm
• Results and comparisons
• Extension: MSAA
• Conclusion / Future work
Goal
• Sparse sampling for reconstruction• ∝ maximum local variations (local bandwidth)
• Adaptive sampling for integration• ∝ variance of the shading integrand
• Multi-resolution rendering and bilateral up-sampling.• a single coarse-to-fine rendering pass
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Local light-field definition
• Spectral Operations
• 2D local bandwidth
• Image-space local bandwidth
• Adaptive sampling for integration
• Validation
• …
Local Light-Field
• 4D local light-field illustrated in 2D
Central ray
Spac
e
Angle
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Local light-field definition
• Spectral Operations
• 2D local bandwidth
• Image-space local bandwidth
• Adaptive sampling for integration
• Validation
• …
Spectral Operations
•Spectral operations [Durand et al. 2005][D
urand et al. 2005]
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Local light-field definition
• Spectral Operations
• 2D local bandwidth
• Image-space local bandwidth
• Adaptive sampling for integration
• Validation
• …
2D Local Bandwidth
• 2D Local Bandwidth• maximum local variations about a light path both in angle and space.
spatial angular
2D local bandwidth
Propagating Local Bandwidth
Propagating Local Bandwidth
• One-bounce 2D bandwidth propagation• Transport (from light to reflector)• Reflection (BRDF & Texture)• Transport (from reflector to pixel)
• Matrix operators on 2D bandwidth
where
Bandwidthat pixel p
Reflectionoperator
Illumination bandwidth
Scale Curvature Mirror BRDF and texture
Curvature ScaleTransport to image plane
Transport to the surface
Input Signals
• Illumination• environment map
• Reflectance• Acquired isotropic BRDFs
• TexturesGold-paint[ Matusik2003]
Grace cathedral [Debevec2001]
Input Signals’ Bandwidth
• Illumination• environment map• Purely angular bandwidth
• Reflectance• Acquired isotropic BRDFs• Purely angular bandwidth
• Textures• Purely spatial bandwidth
Gold-paint[ Matusik2003]
Grace cathedral [Debevec2001]
Computing Local Bandwidth
• Discrete Wavelet Transform to compute bandwidth• environment map and reflectance (BRDF & Texture)
• Discrete Wavelet Transform instead of WFFT • wavelets are well localized both in space and frequency.
Space Frequency
Daubechies 4 tap wavelet
FFT
Local Bandwidth Examples
Gold-paint[ Matusik2003]
Angular bandwidth
Angular bandwidthGrace cathedral [Debevec2001]
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Local light-field definition
• Spectral Operations
• 2D local bandwidth
• Image-space local bandwidth
• Adaptive sampling for integration
• Validation
• …
Image-Space Bandwidth
• We combine bandwidth from various incident directions to get the final bandwidth at a given pixel.
• Taking a max is too conservative, we take the weighted average instead.
• Bandwidth at each pixel
Bandwidth estimate based on max Bandwidth estimate Rendered image
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Adaptive sampling for integration
• Validation
• The rendering algorithm
• Results and comparisons
• Extension: MSAA
• Conclusion / Future work
Variance of the Shading Integrand
• # shading samples ∝ sum of bandwidths, weighted by the illumination times reflectance squared.
• The sum is a conservative approximation of the actual variance of the shading integrand.
Angular bandwidth arriving at the surface
Angular bandwidth of the reflectance
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Adaptive sampling for integration
• Validation
• The rendering algorithm
• Results and comparisons
• Extension: MSAA
• Conclusion / Future work
Validation of the Bandwidth and Variance
Estimated variance Measured varianceEstimated bandwidth Measured bandwidth(WFFT)
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Adaptive sampling for integration
• Validation
• The rendering algorithm
• Results and comparisons
• Extension: MSAA
• Conclusion / Future work
The Algorithm
• Step 1• Compute g-buffers• Load Illumination and reflectance bandwidth
• Step 2• Estimate bandwidth and variance
• Step 3• Mip-map bandwidth and variance
• Step 4• Shade and up-sample
Video: algorithm
QuickTime™ and a decompressor
are needed to see this picture.
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Adaptive sampling for integration
• Validation
• The rendering algorithm
• Results and comparisons
• Extension: MSAA
• Conclusion / Future work
Result (Final Image)
Result (Bandwidth)
Result (Variance)
Timings
• Fast bandwidth/variance estimation• (8 ms)
(ms)(ms)
Timings
• Computation time scales linearly with the total number of shading samples
Comparison with ReferenceOur algorithm
(1015 ms)Similar time
(906 ms)Similar quality
(2639 ms)
Comparison with Spherical Gaussian Approximation (Wang et al. 2009)
Fast but different from ground-truth (notice that color changing effects are missing)
Path-traced reference
Interactive but more accurate
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Adaptive sampling for integration
• Validation
• The rendering algorithm
• Results and comparisons
• Extension: MSAA
• Conclusion / Future work
ExtensionAdaptive Multi-Sample Anti-Aliasing
Video: adaptive MSAA
QuickTime™ and a decompressor
are needed to see this picture.
Timings
• Sub-linear performance for MSAA
Roadmap
• Related work
• Our technique
• The goal
• Sparse sampling for reconstruction
• Adaptive sampling for integration
• Validation
• The rendering algorithm
• Results and comparisons
• Extension: MSAA
• Conclusion / Future work
Conclusion
• Interactively shading dynamic geometry with acquired materials• Shade a fraction of pixels using bandwidth• Adaptively sample shading integrals• Combine shaded pixels using up-sampling
• Exploiting bandwidth to sub-linearly scale MSAA with deferred shading
Future Work
• Local light sources
• Better up-sampling algorithm
• Depth-of-field
• 6D bandwidth computation for SVBRDFs
• Visibility and indirect illumination
Thank you
Interactive rendering of Interactive rendering of acquired materials on dynamic geometryacquired materials on dynamic geometry
using bandwidth predictionusing bandwidth prediction