Non-Linear Kernel-Based Precomputed Light Transport Paul Green MIT Jan Kautz MIT Wojciech Matusik MIT Frdo Durand MIT Henrik Wann Jensen UCSD Geometry & Viewpoint All-Frequency Lighting Rendered Frame Reflectance Interactive High Quality 6D Relighting Precomputed Light Transport Courtesy of Sloan et al Exit Radiance Distant Radiance Incident Radiance Shadowing Inter-reflection Transport function maps distant light to incident light Can Include BRDF if outgoing direction is fixed Light Transport Linear Operator Radiance L o at point p along direction is weighted sum of distant radiance L i Outgoing Radiance Transport Vector Distant Radiance (Environment Map) ExampleExample Transport Function (log scale) Environment Map BRDF Weighted Incident Radiance Exit Radiance (outgoing color) Its a Dot Product Between Lighting and Transport Vectors!! Raw Transport Matrices Enormous 50,000 Vertices 24,200 Element Environment Map 92 View Directions Direct Lighting-Transport product too costly 24,200 multiplies / vertex Direct Evaluation Infeasible 50 GBs raw data But the formula still works in any other basis PLT is Representation Key issue of PLT is representation of Transport and Lighting efficiently. Efficiency of: Storage Lighting-Transport Product Linear Approximation Precomputed Radiance Transfer [Sloan et al 02,03] Low Order Spherical Harmonics Soft Shadows Low Frequency Lighting Not Practical For High Frequency Lighting Courtesy of Ng et al.2003 Non-Linear Approximation Non-Linear Wavelet Lighting Approximation [Ng et al 03] Haar Wavelets Non-Linear Approximation All Frequency Lighting Arbitrary BRDF -> Fixed View Courtesy of Ng et al.2003 What is Non-Linear Approximation? Non-linear: Approximating basis set depends on input Linear: First 5 Coefficients Non-linear: 5 Largest Coefficients SSE = 3,037 SSE = 935 Overview of our method Precompute Transport at a sparse set of sample view directions Backwards Photon Tracing Approximate Transport Non-Linear Parametric Representation Render Fast Lighting-Transport Product View Point Interpolation PrecomputationPrecomputation Azimuth Elevation Transport In spherical coordinates (Lat/Long) Log Scale PrecomputationPrecomputation Specular Diffuse Shadowed Inter-reflections Refracted Backwards photon tracing from fixed view, fixed position p PrecomputationPrecomputation Photon Cloud Latitude / Longitude Map Elevation Angle Azimuthal Angle Density Estimation Photon Cloud After Density Estimation Factoring Transport View Independent (diffuse) Consistent across all views View Dependent (specular) One per view Full Transport RecapRecap p Non-linear Approximation Overview of our method Precompute Transport at a sparse set of sample view directions Backwards Photon Tracing Approximate Transport Non-Linear Parametric Representation Render Fast Integration Method View Point Interpolation p Transport approximated by sum of constant valued box functions Arbitrary weight, size and position Expressive as Haar Wavelets Even More Flexible! Non-Linear Approximation w j weight K j size and position Original Non-linear Approximation Approximated Non-Linear Approximation Box: Arbitrary location & size Our approach Wavelet: Rigid dyadic domain [Ng et al. 03] Non-Linear Approximation SSE = 652 SSE = 935 Non-Linear Wavelet Approximation 5 Largest Coefficients [Ng et al.] Non-Linear Box Kernel Approximation 5 Boxes Our approach Non-Linear Approximation How do you find boxes? It is hard Decomposition is not unique (Non-Orthogonal) Our Approach Greedy Strategy for View Dependent K-d subdivision for View Independent [Matusik et al 04] Non-Linear Approximation View Independent (diffuse) View Dependent (specular) Original Approximated Original Approximated Overview of our method Pre-compute Transport at a sparse set of sample view directions Backwards Photon Tracing Approximate Transport Non-Linear Parametric Representation Render Fast Lighting-Transport Product View Point Interpolation p Rendering Box Kernels Summed Area Table Lookup Exit Radiance (outgoing color) Approximated Transport Lighting Only Pre-computed Transport Functions for sparse set of outgoing directions Interpolate Outgoing Radiance ? Rendering Novel Views p ? Interpolate Parameters Interpolate Box parameters Position Size Weight Drawbacks: Visibility But is at least plausible Shadowing is View Independent Does not need to be interpolated p ? Interpolating Views Example Resulting Color Interpolate Parameters Our Approach Interpolate Radiance Values Standard Linear Fading SummarySummary Compute Transport Matrix T at sparse set of sample view directions Factor T into view dependent and view independent parts Approximate T using Non-Linear Parametric Representation Render by interpolating parameters from closest precomputed Transport Vectors p p ? VideoVideo ContributionContribution Non-Linear Representation View Point Interpolation Technique All-Frequency Relighting From Non-Fixed Viewpoints with Arbitrary Reflectance and Transport Effects Future Work Other Non-Linear Approximations Formal Box Decomposition Method Compression of Transport Vectors Hardware Acceleration Thank You