![Page 1: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/1.jpg)
Deep Scattering: Rendering Atmospheric Clouds withRadiance-Predicting Neural Networks
SIMON KALLWEIT, Disney Research and ETH Zürich et al.
Presenter: MinKu Kang
ACM Transactions on Graphics, Publication date: November 2017
![Page 2: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/2.jpg)
Ambient sound propagation
In Previous Talk from Dennis
![Page 3: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/3.jpg)
edge-darkening effects silverlining
Cloud Rendering
https://www.youtube.com/watch?v=0MJl9IF_3fI
![Page 4: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/4.jpg)
http://ww2010.atmos.uiuc.edu/(Gh)/guides/mtr/opt/mch/sct.rxml
Scattering of Light
Light scattering in
microscale, not just in
macro scale
![Page 5: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/5.jpg)
Problem Configuration & Notation
𝜔: 𝑑𝑖𝑟𝑒𝑐𝑡𝑖𝑜𝑛
𝑥: 𝑙𝑜𝑐𝑎𝑡𝑖𝑜𝑛
We want to know (compute) the radiance at (𝑥, 𝜔)
To render a whole cloud image,
We need to know the radiance at all (visible)
positions and directions
Problem: How to efficiently compute the
radiance at a specific position and a direction ?
![Page 6: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/6.jpg)
Problem Configuration & Notation
𝜔: 𝑑𝑖𝑟𝑒𝑐𝑡𝑖𝑜𝑛
𝑥: 𝑙𝑜𝑐𝑎𝑡𝑖𝑜𝑛
We want to know (compute) the radiance at (𝑥, 𝜔)
To render a whole cloud image,
We need to know the radiance at all (visible)
positions and directions
Problem: How to efficiently compute the
radiance at a specific position and a direction ?
But, there are too many discrete particles to consider
(they are not even polygons!).
Is this possible to use rendering equation we have
learned ?
![Page 7: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/7.jpg)
Radiative Transfer
The radiative transfer equation
Integrating both sides of the differential RTE along ω
: extinction
coefficient
![Page 8: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/8.jpg)
Radiative Transfer
𝜔𝑥
ෝ𝜔
𝑐𝑜𝑛𝑡𝑟𝑖𝑏𝑢𝑡𝑖𝑜𝑛 𝑓𝑎𝑐𝑡𝑜𝑟: 𝜔 ∙ ෝ𝜔
Neighborhood
surface 𝑆2
Boundary
![Page 9: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/9.jpg)
RADIANCE-PREDICTING NEURAL NETWORKS
The in-scattered radiance
Rule out uncollided radiance
(directly from the sun)
This is what the NN
predicts (estimate)
![Page 10: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/10.jpg)
A combination of Monte Carlo integration and neural networks
The in-scattered radiance
This is what the NN
predicts (estimate)
Monte-Carlo Integration
![Page 11: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/11.jpg)
RADIANCE-PREDICTING NEURAL NETWORKS
Want to find (learn) a function
Such that,
given
it predicts
S: shading configuration
around 𝑥,𝜔
![Page 12: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/12.jpg)
RADIANCE-PREDICTING NEURAL NETWORKS
Want to find (learn) a function
via
using
![Page 13: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/13.jpg)
The Descriptor at a specific configuration (𝑥, 𝜔)
• Each descriptor consists of 5 × 5 × 9 stencils
• The stencil at level k is scaled by 2𝑘−1
• They use K=10 levels (10 stenciles)
• Each stencil is formed by 225 points
• The stencil is oriented towards the light source
• Two levels of the hierarchy are shown here
![Page 14: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/14.jpg)
The Descriptor at a specific configuration (𝑥, 𝜔)
The Descriptor:
𝑥: 𝑙𝑜𝑐𝑎𝑡𝑖𝑜𝑛𝜔: 𝑑𝑖𝑟𝑒𝑐𝑡𝑖𝑜𝑛𝜔𝑙: 𝑑𝑖𝑟𝑒𝑐𝑡𝑖𝑜𝑛 𝑡𝑜𝑤𝑎𝑟𝑑𝑠 𝑡ℎ𝑒 𝑙𝑖𝑔ℎ𝑡 𝑠𝑜𝑢𝑟𝑐𝑒
![Page 15: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/15.jpg)
Neural Network Architecture (progressive feeding)
The most finest scale stencil
The most coarse scale
Outout (L)
![Page 16: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/16.jpg)
Ground Truth data from Path Tracing
N = ~15 million samples
Adam update rule using the default learning rate
The minibatches of size |B| = 1000
It requires∼12 h of training on a single GPU
Training Configuration
![Page 17: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/17.jpg)
Path Tracing Radiance-Predicting Neural Networks (RPNN)
Result (Test Time)
![Page 18: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/18.jpg)
Result (Test Time)
They argued that RPNN (seconds to minutes.) converges 24 times faster than PT
![Page 19: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/19.jpg)
Experiment - Neural Network Architecture
Progressive feeding
The entire stencil hierarchy
is input to the first layer
This highlights the benefit of the
progressive feeding that provides
means to better adapt to signals at
different frequency scales.
Validation
error
![Page 20: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/20.jpg)
Experiment – Stencil Size
A good balance between accuracy
and the cost of querying the
density values and number of
trainable parameters in the
network
Validation
error
![Page 21: Deep Scattering: Rendering Atmospheric Clouds with Radiance …sungeui/ICG/Students/CS 482... · 2018. 12. 3. · Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting](https://reader036.vdocument.in/reader036/viewer/2022071503/61238e7940683e17f570dd67/html5/thumbnails/21.jpg)
Summary
Radiative Transfer Equation (RTE)
Hierarchical Stencil Descriptor Progressive Feeding Neural Network