Download - Learning-Based Rendering - KAIST
Learning-Based Rendering
2019.05.16.
20193138 Jaeyoon Kim
1
Review for previous presentation
• “Microfacet Model and Material Appearance” by Hakyeong Kim
2
Machine Learning in Rendering
From CheolMin’s slides From Saehun’s slides3
Deep Scattering:Rendering Atmospheric Clouds withRadiance-Predicting Neural Networks
by S. Kallweit, T. Müller, B. McWilliams, M. Gross, J. Novák
Slides and images from “Deep Learning for Light Transport” by Jan Novák, Disney Research 4
Remind previous presentation
From Hyunsoo’s slides 5
6
7
8
9
Rendering Clouds
𝐿𝐿 𝐱𝐱,𝑤𝑤 = �0
∞𝑝𝑝 𝐲𝐲|𝐱𝐱 𝛼𝛼�
𝑆𝑆𝑝𝑝 �𝑤𝑤|𝑤𝑤 𝐿𝐿 𝐲𝐲, �𝑤𝑤 d�𝑤𝑤d𝑦𝑦
Predict using a network
10
Rendering Clouds
• Scattered direct illumination• Estimated using MC
11
Rendering Clouds
• Scattered direct illumination• Estimated using MC
• Scattered indirect illumination• Predicted using NN
12
Radiance Predicting MLP
• Uses hierarchical stencil for sampling.• for both local details and overall shape
• Feeds into the network.
13
Radiance Predicting MLP
• Uses hierarchical stencil for sampling.• for both local details and overall shape
• Feeds into the network.• One block with two layers for each stencil.• Some FC layers at the end of the network.
14
15
16
Slides and images from “Bayesian online regression for adaptive direct illumination
sampling” by Petr at al. 17
Noise in MC Rendering
18
Noise in MC Rendering
Non-adaptive sampling[Wang et al. 2009]
Reference
19
Noise in MC Rendering
Non-adaptive sampling[Wang et al. 2009]
Adaptive sampling[Donikian et al. 2006]
20
Direct Illumination
Less important
Occluded
21
Light Clustering
22
Direct Illumination
• 𝐿𝐿 𝐱𝐱,𝑤𝑤 = ∫𝐴𝐴 𝐹𝐹 𝐲𝐲 → 𝐱𝐱 → 𝑤𝑤 d𝐲𝐲
• 𝐹𝐹 𝐲𝐲 → 𝐱𝐱 → 𝑤𝑤 = 𝐿𝐿𝑒𝑒 𝐲𝐲 → 𝐱𝐱 𝐵𝐵 𝐲𝐲 → 𝐱𝐱 → 𝑤𝑤 𝑉𝑉 𝐲𝐲 ↔ 𝐱𝐱 𝐺𝐺 𝐲𝐲 ↔ 𝐱𝐱
• 𝐿𝐿 𝐱𝐱,𝑤𝑤 = 𝐹𝐹 𝐲𝐲→𝐱𝐱→𝑤𝑤𝑝𝑝 𝐲𝐲|𝐱𝐱,𝑤𝑤
𝑤𝑤
𝐲𝐲 𝐱𝐱𝐴𝐴
23
Probability Density Function
• 𝑝𝑝 𝐲𝐲|𝐱𝐱,𝑤𝑤 = 𝑃𝑃 𝑐𝑐|𝐱𝐱 𝑃𝑃 𝑙𝑙|𝑐𝑐 𝑝𝑝 𝐲𝐲|𝑙𝑙,𝑤𝑤• 𝑃𝑃 𝑐𝑐|𝐱𝐱 : selecting a light cluster c ∈ C• 𝑃𝑃 𝑙𝑙|𝑐𝑐 : selecting a light 𝑙𝑙 ∈ 𝑐𝑐, 𝑃𝑃 𝑙𝑙|𝑐𝑐 = 𝛷𝛷𝑙𝑙/∑𝑙𝑙’∈𝑐𝑐 𝛷𝛷𝑙𝑙’
• 𝑝𝑝 𝐲𝐲|𝑙𝑙,𝑤𝑤 : selecting a point 𝑦𝑦 ∈ 𝑙𝑙, using standard techniques in [Pharr et al. 2016; Shirley et al. 1996].
• Only need to predict 𝑃𝑃 𝑐𝑐|𝐱𝐱 .
24
Optimal Cluster Selection
• 𝑉𝑉𝑉𝑉𝑉𝑉 𝐿𝐿 𝐱𝐱 = −𝐿𝐿 𝐱𝐱 2 + ∑𝑐𝑐∈𝐶𝐶1
𝑃𝑃 𝑐𝑐|𝐱𝐱 ∫𝐴𝐴𝑐𝑐𝐹𝐹 𝐲𝐲→𝐱𝐱 2
𝑃𝑃 𝑙𝑙|𝑐𝑐 𝑝𝑝 𝐲𝐲|𝑙𝑙d𝐲𝐲
• 𝑚𝑚2,𝑐𝑐 = ∫𝐴𝐴𝑐𝑐𝐹𝐹 𝐲𝐲→𝐱𝐱 2
𝑃𝑃 𝑙𝑙|𝑐𝑐 𝑝𝑝 𝐲𝐲|𝑙𝑙d𝐲𝐲: second moment of 𝐿𝐿𝑐𝑐 𝐱𝐱 = 𝐹𝐹 𝐲𝐲→𝐱𝐱
𝑃𝑃 𝑙𝑙|𝑐𝑐 𝑝𝑝 𝐲𝐲|𝑙𝑙
• 𝑃𝑃𝑜𝑜𝑝𝑝𝑜𝑜 𝑐𝑐|𝐱𝐱 ∝ 𝐿𝐿𝑐𝑐2 𝐱𝐱 + 𝑉𝑉𝑉𝑉𝑉𝑉 𝐿𝐿𝑐𝑐 𝐱𝐱
25
Bayesian Online Regression
• 𝐃𝐃: training data• 𝜃𝜃: parameters• 𝑝𝑝 𝐃𝐃|𝜃𝜃 : a model describing the likelihood of 𝐃𝐃 given 𝜃𝜃
• Maximum likelihood (ML) estimate• Finding 𝜃𝜃 maximizing 𝑝𝑝 𝐃𝐃|𝜃𝜃• Prone to overfit when data is scarce• Gives poor approximations in early stages of rendering
• Maximum a posteriori (MAP) estimate• Finding 𝜃𝜃 maximizing 𝑝𝑝 𝜃𝜃|𝐃𝐃 ∝ 𝑝𝑝 𝐃𝐃|𝜃𝜃 𝑝𝑝 𝜃𝜃• More robust than ML estimate
26
Bayesian Online Regression
• �̂�𝑒 = 𝐿𝐿𝑒𝑒 𝐲𝐲→𝐱𝐱 𝑉𝑉 𝐲𝐲↔𝐱𝐱 cos𝜃𝜃𝐲𝐲𝑑𝑑2 𝐱𝐱,𝐲𝐲 𝑃𝑃 𝑙𝑙|𝑐𝑐 𝑝𝑝 𝐲𝐲|𝑙𝑙
, �̂�𝑒𝐱𝐱 = �̂�𝑒cos𝜃𝜃𝐱𝐱
• Data 𝐃𝐃 consists of tuples �̂�𝑒𝐱𝐱,𝑖𝑖 , �̂�𝑑𝑖𝑖 .
• 𝑝𝑝(𝐃𝐃|𝜃𝜃) = ∏𝑖𝑖𝑁𝑁 𝑝𝑝 �̂�𝑒𝐱𝐱,𝑖𝑖|�̂�𝑑𝑖𝑖 ,𝜃𝜃
• 𝑝𝑝 �̂�𝑒𝐱𝐱|�̂�𝑑,𝜃𝜃 = 𝛿𝛿 �̂�𝑒𝐱𝐱 𝑝𝑝0 + 1 − 𝑝𝑝0 𝑁𝑁 �̂�𝑒𝐱𝐱| 𝑘𝑘�𝑑𝑑2
, ℎ�𝑑𝑑4• 𝜃𝜃 = 𝑝𝑝0, 𝑘𝑘, ℎ• 𝑝𝑝 𝜃𝜃 = 𝐵𝐵 𝑝𝑝0| �𝑁𝑁𝑜𝑜, �𝑁𝑁𝑣𝑣 𝑁𝑁 − 𝛤𝛤−1 𝑘𝑘, ℎ|𝜇𝜇0, �𝑁𝑁, �𝑁𝑁𝛼𝛼 ,𝛽𝛽
27
Bayesian Online Regression
• 𝐿𝐿𝑐𝑐 𝐱𝐱 ≈ 1−𝑝𝑝0 𝑘𝑘�𝑑𝑑2
• 𝑉𝑉𝑉𝑉𝑉𝑉 𝐿𝐿𝑐𝑐 𝐱𝐱 ≈ 1−𝑝𝑝0 𝑝𝑝0𝑘𝑘2+ℎ�𝑑𝑑4
• 𝑃𝑃∗ 𝑐𝑐|𝐱𝐱 ∝ 1�𝑑𝑑2
1 − 𝑝𝑝0 2𝑘𝑘2 + 1 − 𝑝𝑝0 𝑝𝑝0𝑘𝑘2 + ℎ
28
Summary
• Light preprocess (clustering)• During each Next event estimation:
• Obtain cached clustering.• Compute distributions of estimates for each cluster. (mean, variance)• Build distribution over clusters.• Sample direct illumination.• Record new data for sampled cluster.
29
Results
30
Results
31
Reference
• Simon Kallweit, Thomas Muller, Brian McWilliams, Markus Gross, and Jan Novak. 2017. Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting Neural Networks. ACM Trans. Graph. 36, 6, Article 231 (November 2017), 11 pages.
• J. Novák at al. 2018. Deep Learning for Light Transport. Disney Research.
• Petr Vevoda, Ivo Kondapaneni, and Jaroslav Křivánek. 2018. Bayesian online regression for adaptive direct illumination sampling. ACM Trans. Graph. 37, 4, Article 125 (August 2018), 12 pages.
32
Thank you!Any questions?
33
Quiz
Fill in the blanks with words in the box.
1. In “Deep Scattering …“ paper, scattered ( ) illumination is predicted using a neural network.
2. In “Bayesian online regression …” paper, authors aim to reduce noise in ( ) illumination with Bayesian online learning.
(a) Direct(b) Indirect(c) Diffuse(d) Specular
34