thank you for the introduction

59
Mahdi M. Bagher / Cyril Soler / Kartic Subr / Laurent Belcour / Nicolas Holzschuch Mahdi M. Bagher / Cyril Soler / Kartic Subr / Laurent Belcour / Nicolas Holzschuch Interactive rendering of Interactive rendering of acquired materials on dynamic acquired materials on dynamic geometry geometry using bandwidth prediction using bandwidth prediction

Upload: amelia-oliver

Post on 17-Jan-2018

221 views

Category:

Documents


0 download

DESCRIPTION

Shading Direct reflection under distant illumination with no visibility and global illumination effects Rendering an image is computationally expensive. from Wikipedia This paper is about shading, that is computing the amount of illumination arriving at each visible surface point in a virtual scene. That involves computing an integral over all the incident directions for every image pixel. Rendering an image is therefore computationally expensive. Illumination BRDF

TRANSCRIPT

Page 1: Thank you for the introduction

Mahdi M. Bagher / Cyril Soler / Kartic Subr / Laurent Belcour / Nicolas HolzschuchMahdi M. Bagher / Cyril Soler / Kartic Subr / Laurent Belcour / Nicolas Holzschuch

Interactive rendering of Interactive rendering of acquired materials on dynamic geometryacquired materials on dynamic geometry

using bandwidth predictionusing bandwidth prediction

Page 2: Thank you for the introduction

• Direct reflection under distant illumination with no visibility and global illumination effects

• Rendering an image is computationally expensive.

Shading

from

Wik

iped

ia

Illumination BRDF

Page 3: Thank you for the introduction

Acquired Materials

• Using measured materials• photo-realistic rendering• large memory footprint

(33MB for a single isotropic BRDF)

• Not suitable for real-time rendering

Phong shading Measured “Color-Changing-Paint-3”BRDF measurement Gantry [Matusik2002]

Page 4: Thank you for the introduction

Monte Carlo Sampling

• Monte Carlo approximation to the shading integral

• Importance sampling• less noisy

• Data driven reflectance• no analytical importance function• pre-computed Importance samples

Monte Carlo sampling

BRDF importance sampling

Page 5: Thank you for the introduction

Sampling

• Dealing with two types of sampling• Reconstruction• sampling in image space• to render an image

• Integration• sampling in the space of incident directions • to shade each individual pixel

Illumination BRDF Cosine

Page 6: Thank you for the introduction

The Problem

• Shading is slow…

• How to speed-up shading?

Page 7: Thank you for the introduction

Case Study

• What difference various materials make in rendering an image?

Measured reflectance data from [Matusik2002]

Page 8: Thank you for the introduction

Diffuse Materials

• Diffuse materials• shading varies slowly across the image (low frequencies)• many integration samples for each pixel (wide BRDF lobe)

Measured Teflon [Matusik2002] BRDF Lobe [BRDFLab]

Page 9: Thank you for the introduction

Specular Materials

• Specular materials• shading varies quickly across the image (high frequencies) • few integration samples for each pixel (narrow BRDF lobe)

Measured color-Changing-Paint3 [Matusik2002] BRDF Lobe [BRDFLab]

Page 10: Thank you for the introduction

The Problem

• Rendering is slow…• How to speed-up shading?

• Can we exploit the coherency between • Reconstruction samples…• Integration samples…

• …for adaptive sampling?• How to measure the coherency

between samples?

Page 11: Thank you for the introduction

Intuition

• Study the frequencies in the image• As a function of what happens to the light• Traveling from the light source to the camera

Page 12: Thank you for the introduction

The Traveling Light

• Light when traveling is affected by• Travel through free space• Object’s curvature• Reflectance• BRDF• Texture

• Occlusion

Page 13: Thank you for the introduction

Our Contribution

• How to speed-up shading?By sparse sampling the image• How to combine the sparse samples?• Multi-resolution rendering and bi-lateral up-sampling

By adaptive sampling for integration

Page 14: Thank you for the introduction

Our Contribution (Illustrated)

Sparse reconstruction samples Adaptive Integration samples

,

Final shaded image

Multi-resolution shading and bi-lateral up-sampling

∝ the average variance of the shading integrand∝ the local maximum variations in the image

Page 15: Thank you for the introduction

Assumptions

• Direct reflection under distant illumination

• No visibility and global illumination effects

• Allow dynamic editable geometry

• Screen-space rendering in the context of deferred-shading

• Any shading model is possible

• Save more if shading cost is higher e.g. measured materials

• BRDF importance sampling

Normal

Page 16: Thank you for the introduction

Dynamic Geometry Animation

QuickTime™ and a decompressor

are needed to see this picture.

Page 17: Thank you for the introduction

Roadmap

• Related work

• Our technique

• The goal

• Sparse sampling for reconstruction

• Adaptive sampling for integration

• Validation

• The rendering algorithm

• Results and comparisons

• Extension: MSAA

• Conclusion / Future work

Page 18: Thank you for the introduction

Related Work GPU Rendering of Complex Materials

[Heidrich&Seidel1999]

[Ramamoorthi2002]

[Kautz2002]

[Claustres2007]

[Kautz&McCool1999]

[Latta&Kolb2002][McCool2001]

[Wang2009]

Page 19: Thank you for the introduction

Related WorkMulti-Resolution Screen-Space Algorithms

[Nichols&Wyman2009] [Shopf2009][Nichols&Wyman2010]

[Soler2010]

[Nichols2010][Segovia2006]

Page 20: Thank you for the introduction

Related WorkPre-Computed Transport

[Sloan2002]

[Ramamoorthi2009]

[Sun2007]

[Liu2004]

[Loos2011]

[Loos2012]

Page 21: Thank you for the introduction

Roadmap

• Related work

• Our technique

• The goal

• Sparse sampling for reconstruction

• Adaptive sampling for integration

• Validation

• The rendering algorithm

• Results and comparisons

• Extension: MSAA

• Conclusion / Future work

Page 22: Thank you for the introduction

Goal

• Sparse sampling for reconstruction• ∝ maximum local variations (local bandwidth)

• Adaptive sampling for integration• ∝ variance of the shading integrand

• Multi-resolution rendering and bilateral up-sampling.• a single coarse-to-fine rendering pass

Page 23: Thank you for the introduction

Roadmap

• Related work

• Our technique

• The goal

• Sparse sampling for reconstruction

• Local light-field definition

• Spectral Operations

• 2D local bandwidth

• Image-space local bandwidth

• Adaptive sampling for integration

• Validation

• …

Page 24: Thank you for the introduction

Local Light-Field

• 4D local light-field illustrated in 2D

Central ray

Spac

e

Angle

Page 25: Thank you for the introduction

Roadmap

• Related work

• Our technique

• The goal

• Sparse sampling for reconstruction

• Local light-field definition

• Spectral Operations

• 2D local bandwidth

• Image-space local bandwidth

• Adaptive sampling for integration

• Validation

• …

Page 26: Thank you for the introduction

Spectral Operations

•Spectral operations [Durand et al. 2005][D

urand et al. 2005]

Page 27: Thank you for the introduction

Roadmap

• Related work

• Our technique

• The goal

• Sparse sampling for reconstruction

• Local light-field definition

• Spectral Operations

• 2D local bandwidth

• Image-space local bandwidth

• Adaptive sampling for integration

• Validation

• …

Page 28: Thank you for the introduction

2D Local Bandwidth

• 2D Local Bandwidth• maximum local variations about a light path both in angle and space.

spatial angular

2D local bandwidth

Page 29: Thank you for the introduction

Propagating Local Bandwidth

Page 30: Thank you for the introduction

Propagating Local Bandwidth

• One-bounce 2D bandwidth propagation• Transport (from light to reflector)• Reflection (BRDF & Texture)• Transport (from reflector to pixel)

• Matrix operators on 2D bandwidth

where

Bandwidthat pixel p

Reflectionoperator

Illumination bandwidth

Scale Curvature Mirror BRDF and texture

Curvature ScaleTransport to image plane

Transport to the surface

Page 31: Thank you for the introduction

Input Signals

• Illumination• environment map

• Reflectance• Acquired isotropic BRDFs

• TexturesGold-paint[ Matusik2003]

Grace cathedral [Debevec2001]

Page 32: Thank you for the introduction

Input Signals’ Bandwidth

• Illumination• environment map• Purely angular bandwidth

• Reflectance• Acquired isotropic BRDFs• Purely angular bandwidth

• Textures• Purely spatial bandwidth

Gold-paint[ Matusik2003]

Grace cathedral [Debevec2001]

Page 33: Thank you for the introduction

Computing Local Bandwidth

• Discrete Wavelet Transform to compute bandwidth• environment map and reflectance (BRDF & Texture)

• Discrete Wavelet Transform instead of WFFT • wavelets are well localized both in space and frequency.

Space Frequency

Daubechies 4 tap wavelet

FFT

Page 34: Thank you for the introduction

Local Bandwidth Examples

Gold-paint[ Matusik2003]

Angular bandwidth

Angular bandwidthGrace cathedral [Debevec2001]

Page 35: Thank you for the introduction

Roadmap

• Related work

• Our technique

• The goal

• Sparse sampling for reconstruction

• Local light-field definition

• Spectral Operations

• 2D local bandwidth

• Image-space local bandwidth

• Adaptive sampling for integration

• Validation

• …

Page 36: Thank you for the introduction

Image-Space Bandwidth

• We combine bandwidth from various incident directions to get the final bandwidth at a given pixel.

• Taking a max is too conservative, we take the weighted average instead.

• Bandwidth at each pixel

Bandwidth estimate based on max Bandwidth estimate Rendered image

Page 37: Thank you for the introduction

Roadmap

• Related work

• Our technique

• The goal

• Sparse sampling for reconstruction

• Adaptive sampling for integration

• Validation

• The rendering algorithm

• Results and comparisons

• Extension: MSAA

• Conclusion / Future work

Page 38: Thank you for the introduction

Variance of the Shading Integrand

• # shading samples ∝ sum of bandwidths, weighted by the illumination times reflectance squared.

• The sum is a conservative approximation of the actual variance of the shading integrand.

Angular bandwidth arriving at the surface

Angular bandwidth of the reflectance

Page 39: Thank you for the introduction

Roadmap

• Related work

• Our technique

• The goal

• Sparse sampling for reconstruction

• Adaptive sampling for integration

• Validation

• The rendering algorithm

• Results and comparisons

• Extension: MSAA

• Conclusion / Future work

Page 40: Thank you for the introduction

Validation of the Bandwidth and Variance

Estimated variance Measured varianceEstimated bandwidth Measured bandwidth(WFFT)

Page 41: Thank you for the introduction

Roadmap

• Related work

• Our technique

• The goal

• Sparse sampling for reconstruction

• Adaptive sampling for integration

• Validation

• The rendering algorithm

• Results and comparisons

• Extension: MSAA

• Conclusion / Future work

Page 42: Thank you for the introduction

The Algorithm

• Step 1• Compute g-buffers• Load Illumination and reflectance bandwidth

• Step 2• Estimate bandwidth and variance

• Step 3• Mip-map bandwidth and variance

• Step 4• Shade and up-sample

Page 43: Thank you for the introduction

Video: algorithm

QuickTime™ and a decompressor

are needed to see this picture.

Page 44: Thank you for the introduction

Roadmap

• Related work

• Our technique

• The goal

• Sparse sampling for reconstruction

• Adaptive sampling for integration

• Validation

• The rendering algorithm

• Results and comparisons

• Extension: MSAA

• Conclusion / Future work

Page 45: Thank you for the introduction

Result (Final Image)

Page 46: Thank you for the introduction

Result (Bandwidth)

Page 47: Thank you for the introduction

Result (Variance)

Page 48: Thank you for the introduction

Timings

• Fast bandwidth/variance estimation• (8 ms)

(ms)(ms)

Page 49: Thank you for the introduction

Timings

• Computation time scales linearly with the total number of shading samples

Page 50: Thank you for the introduction

Comparison with ReferenceOur algorithm

(1015 ms)Similar time

(906 ms)Similar quality

(2639 ms)

Page 51: Thank you for the introduction

Comparison with Spherical Gaussian Approximation (Wang et al. 2009)

Fast but different from ground-truth (notice that color changing effects are missing)

Path-traced reference

Interactive but more accurate

Page 52: Thank you for the introduction

Roadmap

• Related work

• Our technique

• The goal

• Sparse sampling for reconstruction

• Adaptive sampling for integration

• Validation

• The rendering algorithm

• Results and comparisons

• Extension: MSAA

• Conclusion / Future work

Page 53: Thank you for the introduction

ExtensionAdaptive Multi-Sample Anti-Aliasing

Page 54: Thank you for the introduction

Video: adaptive MSAA

QuickTime™ and a decompressor

are needed to see this picture.

Page 55: Thank you for the introduction

Timings

• Sub-linear performance for MSAA

Page 56: Thank you for the introduction

Roadmap

• Related work

• Our technique

• The goal

• Sparse sampling for reconstruction

• Adaptive sampling for integration

• Validation

• The rendering algorithm

• Results and comparisons

• Extension: MSAA

• Conclusion / Future work

Page 57: Thank you for the introduction

Conclusion

• Interactively shading dynamic geometry with acquired materials• Shade a fraction of pixels using bandwidth• Adaptively sample shading integrals• Combine shaded pixels using up-sampling

• Exploiting bandwidth to sub-linearly scale MSAA with deferred shading

Page 58: Thank you for the introduction

Future Work

• Local light sources

• Better up-sampling algorithm

• Depth-of-field

• 6D bandwidth computation for SVBRDFs

• Visibility and indirect illumination

Page 59: Thank you for the introduction

Thank you

Interactive rendering of Interactive rendering of acquired materials on dynamic geometryacquired materials on dynamic geometry

using bandwidth predictionusing bandwidth prediction